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PKEFACt. 


Ttie  National  Bureau  of  Standards  is  tne 
primary  technical  advisor  on  computers  tor 
government  agencies  (Brooks  Act,  Public  Law 
&9-3k)6)  .  NBb  assists  Federal  agencies  in  specitic 
projects,  ana  so  obtains  first-hand  experience 
with  agency  problems  and  competencies.  This  guide 
was  prepared  drawing  upon  extensive  contacts  with 
government  computing  personnel,  and  using  the 
published  recommendations  of  authorities  m 
software  management.  A  variety  of  executive 
guides  and  primers  in  other  fields  were  reviewed 
in  choosing  the  scope  and  approach  to  the  material 
covered  here.  uther  staff  members  ot  trie 
Institute  for  Computer  Sciences  and  Technology 
assisted  by  reviewing  the  material  as  it  was 
developed.  The  author  was  helped  particularly  by 
the  constructive  comments  of  I.  Trotter  Hardy  ana 
Theoaore  Linden. 


NOTICE  TO  READERS 


This  guide  is  intended  to  become  a  standard 
reference  for  managing  Federal  government  software 
projects.  Comments  by  the  readers  on  its 
completeness,  clarity,  etc.  would  be  valuable  in 
making  appropriate  revisions  for  this  purpose.  A 
postage-free  reader  evaluation  form  is  included  at 
the  back  of  the  report,  and  you  are  encouraged  to 
complete  and  return  it. 
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COMPUTER  SOFTWARE  MANAGEMENT: 
A  primer  for  project  management 
and  quality  control 


Dennis  w.  Fife 


Today,  providing  computer  software  involves 
greater  cost  and  risK  than  providing  computer 
equipment,  because  Hardware  is  mass  produced  by 
industry  using  proven  technology,  wnile  sottware 
IS  still  produced  mostly  Dy  tne  craft  of 
individual  computer  programmers.  This  brief  guide 
IS  intended  tor  managers  who  are  responsible  for 
computer  projects,  to  explain  the  use  of  quality 
controls  and  software  management  methods.  The 
typical  problems  of  software  development  are 
explained.  Over  twenty  distinct  quality  controls 
are  defined,  and  recommendations  are  given  for 
software        management  actions.  Empirical 

information  is  included  that  would  help  top 
executives  to  appreciate  the  potential  problems 
and   importance  of  software  management. 


Keywords:         computer  management;  computer 

programming;     computer     project     control;  computer 

software;  software  engineering;  software  quality; 
software  reliability. 


Whi   Trilb   oUlbE  wAb  WRITTEN 


Computer  software  is  the  computer  instructions  or 
programs,  as  well  as  the  data  files  and  descriptive  manuals, 
that  nave  to  be  provided  to  maKe  computer  equipment 
(hardware)  perform  useful  services.  Today,  providing 
software  involves  greater  cost  and  risk  than  providing  the 
equipment,  because  hardware  is  mass  produced  by  industry 
using  proven  technology,  while  software  is  still  produced 
mostly  by  the  craft  of  individual  computer  programmers  and 
users. 
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Software  management  is  the  technical  and  management 
control  throughout  the  life  cycle  of  all  software  that 
supports  an  organization's  mission  and  objectives.  Software 
management  is  effective  when  high  quality  software  is  being 
obtained  with  reasonable  and  controllable  costs.  Software 
management  is  inadequate  when  poor  decisions  and  lax 
technical  controls  result  in  faulty  software  performance, 
computer  service  fiascos,  or  project  cost  overruns.  The 
unabated  high  cost  of  software  as  well  as  many  publicized 
cases  of  software  catastrophes  are  convincing  evidence  that 
software  management  is  poorly  understood  and  practiced  in 
most  organizations. 

The  problem  is  deep-seated  and  serious,  because  many 
managers  who  control  software  projects  do  not  have 
professional  computing  expertise,  and  many  programmers  lack 
the  technical  sKills  to  properly  design  and  produce  the 
Highly  complex  computer  software  needed  today.  Equally 
important,  the  expansion  of  computer  applications  over  the 
last  twenty  years  has  far  outpaced  the  evolution  of  basic 
principles  and  engineering  discipline  in  software  design  and 
proQuction.  Much  more  applied  research  is  needed  before 
software  engineering  will  progress  to  the  effectiveness 
found   in  other  engineering  and  production  fields. 


WHO  THIS  GUIDE   IS  FOR 


This  guide  is  intended  for  managers  and  staff  who  are 
responsible  for  computer  projects.  These  will  be  data 
processing  professionals  and  supervisors  primarily.  But  the 
intended  audience  also  includes  executives  and  managers  who 
were  not  trained  as  computer  professionals  and  who  perhaps 
have  never  used  a  computer  themselves.  Every  reader  will 
need  to  understand  the  rudiments  of  computer  operation  and 
programming. 


WHAT  THIS  GUIDE   IS  FUR 


The  objective  of  this  guide  is  to  give  the  reaaer  an 
appreciation  of  the  software  pitfalls  in  computer  projects, 
and  to  explain  tne  use  of  quality  controls  ana  software 
management  methods.  The  typical  problems  of  software 
production  are  highlighted,  and     recommendations     are  given 
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for  setting  up  software  management  procedures. 


AriAT  This   bUIDL   IS   NUT  FOK 


The  brevity  ana  special  purpose  of  tnis  guide  mean  tnat 
It  is  unsuitable  for  otner  important  purposes.  In 
particular,  tnis  guide  is  NOT  a  survey  of  programming 
methods,  NOK  a  handbook  for  the  direct  supervision  of 
computer  programmers.  Its  use  cannot  compensate  for  a  lack 
of  technical  expertise  that  is  essential  to  direct 
management  of  software  design,  development,  and  maintenance. 
Finally,  this  guide  is  NOT  a  popular  treatment  to  arouse  the 
interest  and  concern  of  the  general  public.  It  is  expected 
that  readers  have  a  professional  interest  and  obligation  in 
decision-making  for  software  projects. 


HOW   TO   USE  THIS  GUIDE 


The  sections  marked  by  (#)  in  the  Table  of  Contents 
contain  specific  recommendations  and  guidelines  for 
effective  software  management.  This  material  should  be 
utilized  to  develop  appropriate  administrative  oroers  or 
management  procedures  to  implement  the  recommended  practices 
in  all  software  projects.  Selected  references  for  further 
stuay  of  the  more  important  topics  are  given  at  the  end  of 
this  guide.  Aaaitional  sections  of  this  guide  contain 
information  that  woula  benefit  a  senior  executive  in 
unaer stand ing  the  problems  ana  importance  of  software 
management. 


WHY  SOFTWARE  MANAGEMENT   IS  IMPORTANT 


♦Software  is  a  very  expensive  resource  for  government. 

The  procurement  and  development  of  computer  software  is 
"big  business"  for  many  organizations,  including  the 
government.  Estimates  of  Federal  government  spending  range 
as  high  as  as  $5  billion  annually  on  software,  and  the 
accumulated  investment  in  current  Federal  software  probably 
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exceeds  $25  billion  dollars.  Software  costs  greatly  exceed 
equipment  costs  over  the  useful  life  of  computer  services, 
and  the  initial  development  costs  for  government  software 
systems  are  growing  to  enormous  levels.  For  instance,  cost 
estimates  for  the  newly  proposed  Tax  Administration  System 
of  the  Internal  Revenue  Service  are  well  over  $500  million 
dollars,  and  the  Air  Force  has  estimated  its  software  costs 
for  command  and  control  systems  in  the  1980  's  will  be 
several  billion  dollars. 

*The  government  software  inventory  is  large  and  complex. 

Among  the  more  than  8,000  government  computer 
installations,  many  have  500  or  more  programs  for  specific 
government  tasks,  in  addition  to  others  acquired  from  the 
computer  manufacturer  for  general  support  of  computer 
operation.  Government  computer  applications,  which  include 
sophisticated  military  systems  as  well  as  public  service 
systems  handling  millions  of  accounts,  are  typical  of  the 
most  challenging  uses  of  computer  technology  within  the 
sta te-of -the-ar  t . 


*Software  flaws  and  mismanagement  can  be  ruinous  to 
government  projects  and  public  services,  and  even  to 
national  goals. 

Government  and  business  are  completely  dependent  on 
computer  services  today.  Manual  methods  are  no  longer 
considered  because  human  means  cannot  be  organized  to  handle 
the  volume  of  data  and  short  deadlines  that  are  needed  in 
widespread  public  services.  Social  and  economic  activity  is 
geared  to  the  performance  of  computer  systems.  Examples  are 
many,  as  in  airline  reservations  systems  or  the  Social 
Security  Administration  systems  that  distribute  over  $50 
billion  annually  to  about  25  million  individuals.  These 
benefits  are  possible  only  with  effective  management  of  the 
computer  software.  In  recent  years,  recurring  cases  of 
major  software  failures  have  demonstrated  the  serious 
consequences  of  software  mismanagement  to  legislators, 
journalists,  and  the  public.  One  dramatic  case  occurred  in 
1975,  when  a  government  agency  made  more  than  $500  million 
in  overpayments  to  individuals  as  a  result  of  erroneous 
computer  programs. 

*No  technology  or  standard  practice  now  exists  that  will 
surely  prevent  faulty  design,  logical  errors,  cost  overrun, 
or  late  delivery  for  any  software  project. 

The  preparation  and  maintenance  of  computer  software  is 
a  laborious,  human  task  that  can  require  the  coordinated 
effort  of  dozens     or     even     hundreds     of     programmers.  The 
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possibilities  for  program  errors,  poor  design,  or  poor 
decision-making  may  be  higher  than  in  any  other  technical 
field,  because  of  extreme  technical  complexity  and  the  lack 
of  standards  and  common  practices  in  this  new  technology. 
The  speed  ana  volume  of  data  handling  by  the  computer 
creates  disastrous  consequences  from  errors  in  programs  and 
accentuates  the  difficulties  caused  by  only  minor  design 
flaws.  Tnere  is  no  practical  method  Known  that  will 
guarantee  software  as  free  from  errors  or  design  flaws. 
Althougn  programming  productivity  has  been  improving,  the 
aavances  have  not  kept  pace  with  the  tremendous  growth  m 
computer  applications  ana  the  aramatic  gains  in  performance 
versus  cost  for  computer  hardware  over  the  past  twenty 
years.  There  are  very  few  standard  computer  programs  for 
common  applications.  There  are  no  standards  covering  common 
work  practices  and  technical  goals,  such  as  the  measurement 
of  software  reliability.  Thus,  every  software  project  is  a 
unique  effort.  Success  or  failure  is  determined  more  by  the 
individual  technical  skills  and  management  capability  of  the 
personnel   involved,   than  by  any  other  factor. 


WHY   EVERY  AGENCY   NEEDS   SOFTWARE  MANAGEMENT 


♦Software  management  must  be  the  responsibility  of  the 
computer   user  organization. 

Over  7S%  of  all  software  is  developed  by  the  users  of 
computers,  rather  than  the  manufacturers  of  computer 
equipment  or  the  producers  of  commercial  software.  Although 
the  computer  industry  is  working  to  improve  software 
management  for  its  products,  industry  cannot  be  expected  to 
provide  a  comprenensive  solution,  nor  to  cover  the  unique 
circumstances  of  each  computer  user, 

♦Software  design  and  programming  is  a  complex  technical 
activity  that  must  be  directed  effectively. 

Although  many  people  nowadays  may  understand  the 
rudiments  of  programming  from  college  or  even  high  school 
courses,  the  simple  homework  exercises  in  such  courses  do 
not  represent  the  "real  world"  of  professional  practice. 
The  design  and  production  of  numerous  programs  that  must 
work  together  for  a  major  service,  such  as  the  payroll  of  a 
large  business,  is  not  routine  effort  that  can  go 
unsupervised  or  be  done  by  junior  personnel.  The  funds  and 
personnel  resources  involved  may  be  a  major  investment  for 
the    organization,  and  the  technical  choices  may  drastically 
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affect  non-computer  work  processes  in  management,  clerical, 
and  client  relations  areas.  Technically  complex  programs 
may  demand  advanced  professional  SKill  which  is  very  scarce 
and  must  be  carefully  airected  at  the  critical  technical 
problems.  Management  and  technical  control  by  a 
professional  software  designer  is  essential  for  resolving 
complex  design  issues  and  for  directing  programmers  to 
produce  a  specified  product  within  cost  and  schedule 
targets . 

♦Software  management  establishes  accountability  for  project 
decisions  and  objectives,  and  provides  upper  management  with 
visible  measures  of  progress  toward  desired  goals. 

Software  projects  usually  are  initiated  by  managers  in 
such  areas  as  payroll,  engineering,  plant  and  supply,  etc. 
Although  they  may  control  the  project  budget  and  schedule, 
and  have  the  best  information  on  potential  benefits  and 
requirements,  they  are  often  unprepared  to  conduct  a  design 
project  or  to  judge  the  results  provided  by  an  outside 
technical  group.  Thus,  software  designers  sometimes  worK 
independently  using  their  own  judgement,  with  little 
pressure  to  reliably  estimate  their  progress  and  the 
remaining  work  or  costs.  Software  management  mcluaes 
cneckpoints  to  assure  that  important  objectives  and  issues 
are  carefully  evaluated  by  the  concerned  parties,  especially 
m  regard  to  cost ,  qual i ty ,  and  scneaule,  and  tnat  users  and 
designers  take  their  appropriate  responsibility  for  tne 
decisions  made. 

♦Software  management  provides  the  technical  controls  and 
resource  management  to  produce  high  quality  software  within 
acceptable  funding  and  time  limits. 

Software  management  emphasizes  technical  criteria  and 
working  procedures  to  resolve  the  technical  issues  that  are 
critical  to  the  success  or  failure  of  a  project.  Major 
questions  commonly  arise  concerning  the  feasibility  or 
performance  of  a  desired  system,  the  attainable  quality  of 
the  system  considering  the  resources  available  to  produce 
it,  and  the  methodology  for  design  and  testing  in  order  to 
achieve  the  most  effective  software  product.  Software 
management  delineates  quality  controls,  standards,  planning 
factors,  and  experience  data  which  help  designers  and 
programmers  to  organize  and  direct  their  efforts  efficiently 
in  solving  such  problems  within  available  cost  and  time 
1 imits . 

♦Software  management  controls  costs  after  software  operation 
beg  ins . 


The  useful  life  of  software  for  a  significant  service 
typically  is  over  eight  years.  Experience  has  shown  that 
continual  changes  are  necessary  in  software,  to  correct 
previously  unknown  errors,  to  improve  useability  or 
performance,  or  to  include  new  functions  required  because  of 
legal,  administrative,  or  technical  changes.  As  much  as  70% 
of  software  cost  occurs  in  this  redesign  and  maintenance, 
after  software  has  been  delivered  and  put  into  use.  It  is 
important  to  recognize  that  maintenance  is  no  less 
cnallenginy  than  development,  for  change  to  programs 
involves  new  aesign  work  ana  snould  be  aone  accoramg  to 
planned  cost  and  time  goals.  software  management 
t^roceaures,  consistently  appliea  throughout  the  sottware 
life,  will  maKe  best  use  of  limited  funas  by  stressing  close 
management  of  effort  toward  tne  most  needea  sottware 
capabil 1 t les . 


BASIC   TASKS   OF  SOFTWARE  MANAGEMENT 


♦Software  management  consists  of  all  the  technical  and 
management  activities,  decisions,  and  controls  that  are 
directly  required  to  purchase,  produce,  or  maintain  software 
throughout  the  useful  life  of  a  computer  system  or  service. 

Software  management  is  primarily  concerned  with 
computer  programs,  data  files,  and  documentation,  rather 
than  the  computer  equipment  configuration  that  executes  the 
programs.  Even  so,  software  designers  must  have  a  say  in 
nardware  decisions  if  they  are  to  meet  the  goals  for  a 
computer  service.  The  term  "system"  will  be  used  here  for 
the  combination  of  hardware  and  software  that  provides  a 
computer  service.  Since  software  has  little  purpose  without 
haraware,  the  terms  "system"  ana  "software"  may  be  used 
in terchangeaoly  without  causing  confusion. 

ine  basic  activities  ana  aecisions  are  best  introducea 
in  relation  to  tne  commonly  recognizea  pnases  of  a  system's 
life  cycle,  snown  in  tigure  1.  Tne  INITIATION  pnase 
involves  the  assessment  of  an  existing,  inadequate  system, 
in  order  to  determine  the  feasioility  of  replacing  it  witn 
an  improved  system.  The  four  pnases,  DEFINITION  through 
TESTING,  are  the  development  period,  where  a  new  or  improved 
software  product  is  being  designed  and  implemented.  The 
OPERATION  phase  starts  with  tne  operation  of  a  new  system  by 
Its  intended  users,  and  includes  the  subsequent  maintenance 
and  minor  redesigns  that  user  experience  may  dictate. 
Eventually,       maintenance     no     longer     suffices     for  needed 
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improvements,  and  another  development  cycle  is  begun. 

Specitic  guidelines  for  each  phase  will  be  given  after 
the  following  summary  of  the  tasks  included.  Figure  2  helps 
to  understand  each  phase  by  summarizing  the  documents 
produced  that  are  needed  for  quality  control. 

*In  the  INITIATION  phase,  software  management  assures  that 
the  technical  designers  understand  the  purpose  and  scope  of 
the  envisioned  system,  and  adequately  perceive  its  required 
functions . 

The  INITIATION  phase  begins  when  management  recognizes 
that  a  need  exists  for  a  new  or  improved  computer  service, 
and  that  a  study  should  be  made  to  formulate  a  software 
solution.  During  this  phase,  software  planners  or  system 
analysts  identify  the  intended  users  and  investigate  their 
work  processes.  The  planners  describe  the  services  needed 
from  the  envisioned  software,  and  then  conceive  and  outline 
alternative  software  solutions.  A  close  working 
relationship  with  the  intended  users  is  essential.  The 
alternative  approaches  should  be  described  in  basic  terms 
that  users  can  understand.  User  concurrence  with  the  stated 
requirements  and  the  recommended  software  approach  is 
mandator  y . 

*The  INITIATION  phase  assures  that  the  recommended  software 
approach  is  technically  and  economically  feasible. 

An  estimate  must  be  made  of  the  costs  and  benefits  for 
eacn  of  the  potential  software  solutions.  This  will  lead  to 
a  recommended  solution  and  provide  data  regarding 
feasibility.  Comparable  existing  systems  are  investigated 
to  derive  cost  data  and  to  confirm  that  proposed  solutions 
are  technically  realizable.  Any  pitfalls,  failures,  or 
questionable  design  areas  in  comparable  projects  should  be 
covered  thoroughly.  Comparisons  of  the  proposed  project  to 
past  failures  or  problem  areas  would  assure  that  they  can  be 
avoided  or  resolved.  General  specifications  must  be  written 
to  show  clearly  the  scope  and  character  of  the  recommended 
solution.  The  INITIATION  phase  concludes  with  a  report 
delineating  the  recommended  software  and  the  supporting 
analysis.  Management  may  then  approve  the  concept  and 
budget  the  recommended  funds  and  personnel  for  acquisition 
and  operation  of  the  planned  system. 

*The  DEFINITION  phase  defines  the  functional  and  performance 
requirements,  in  order  to  confirm  that  the  software,  when 
built,  would  meet  its  objectives. 
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The  DEFINITION  phase  begins  when  management  commits 
resources  to  the  design  or  purchase  of  the  proposed 
software.  In  this  phase,  detailed  specifications  are 
created  for  those  externally  apparent  functions  and  design 
characteristics  that  are  crucial  to  the  purpose  and 
operation  of  the  software  in  the  intended  environment. 
Input  ana  output  data,  processing  operations,  and 
performance  goals  are  defined.  Tne  Functional  Requirements 
define  wnat  the  software  must  do  ana  the  general  aspects  of 
how  It  will  oe  constructea,  rather  than  the  details.  The 
DLFIWITIUN  pnase  concluaes  with  a  specification  that  is 
sufficiently  precise  for  outsiae  contracting,  if  uesirea. 
Equally  important,  the  spec ii ica tion  allows  the  intended 
users  to  confirm  that  tne  proposed  system  will  meet  tne  need 
ana  will  nave  acceptaole  qualities  of  useability, 
generality,  etc. 

♦The  DEFINITION  pnase  provides  a  project  plan  for  managing 
the  acquisition  and  installation  of  the  system. 

A  thorough  plan  for  producing  or  purchasing  the 
software  is  prepared  as  part  of  the  DEFINITION  phase. 
Projections  are  made  of  the  detailed  costs.  A  delivery 
schedule  is  laid  out,  including  intermediate  milestones  and 
technical  reviews  for  evaluating  progress  and  insuring  that 
the  project  will  meet  its  goals.  Working  methods  are 
defined  for  quality  control  of  intermediate  and  final 
products.  The  plan  should  extend  to  the  installation  and 
initial  operation  of  the  system,  and  should  include  user 
training,  document  preparation  and  review,  acceptance 
testing,  and  the  continuing  obligations  of  any  contractors, 
vendors,  or  support  organizations.  The  Project  Plan  is 
extremely  important  oecause  it  specifies  how  the  effort  will 
be  managea,  how  problems  will  be  resolvea,  and  how  the 
quality  of  the  final  system  will  be  assurea. 
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Figure  1.     Ttie  Lite  Cycle  ot  a  software  irroauct 
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Figure  2.     Chart  of  Basic  Quality  Control  Documents 


Life  Cycle  Phase 

Documents 

General  Content 

Initiation 

Feasibility  Study  Report 

Requirements  analysis,  definition  and  evaluation 
of  alternative  solutions,  recommended  software 
concept. 

Definition 

Functional  Requirements 

Quality  and  Performance 
Requi  rements 

Project  Plan 

External  specification  of  software  functions  and 
operation,  including  design  constraints. 

Desired  design  attributes  and  quantitative  perform- 
ance parameters  for  key  processing  operations. 

Cost  and  work  breakdown,  detailed  schedule  including 
quality  assurance  milestones,  and  definition  of 
primary  management  and  quality  control  methods. 

Design 

Design/Coding  Standards 
Design  Specifications 
Test  Plan 

Detailed  design  attributes  and  coding  conventions 
for  qual ity . 

Software  architecture,  module  definitions,  interface 
specifications,  and  programming  requirements. 

Testing  criteria,  testing  schedule  and  work  breakdown, 
standard  test  case  specifications. 

Programming 

Review  Reports 

Unit  Test  Reports 
Inspection  Reports 
Software  Manuals 
Program  Listings 

Identified  discrepancies  and  recommendations  from 
team  reviews. 

Results  of  unit  test  for  each  module. 
Results  of  inspection  sessions  for  each  subsystem. 
Usage,  operation  and  maintenance  descriptions. 
Source  Language  statements  for  each  tested  module. 

Testing 

Test/Fault  Report 

Results  of  scheduled  tests. 

Operati  on 

Faul t  Report 
Specification 
Maintenance  Plan 

•swmn-t-nmc      r  1  rri jm*; ffl nr P";   nf  dP'f"Prted   or  SUSDeCted 

failure. 

Functional  and  design  specifications  for  maintenance 
and  improvement  work. 

Work  breakdown,  cost  estimate,  and  schedule  for 
proposed  rework. 
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*Tne  DtiSluN  pnase  produces  a  tliorough  design  spec  it  ica  t  ion 
tor  guiding  a  timely  ana  pr ODlem-f r ee  implementation  ot  the 
proposed  sottware. 

The  DtSIbW  phase  begins  when  the  Functional 
requirements  have  been  validated  by  intended  users,  and 
implementation  of  the  system  is  approved.  The  DbSIbiM  phase 
results  in  detailed  specifications  of  the  internal 
construction  of  the  software,  for  use  by  programmers  who 
will  implement  the  design.  The  DESIGN  phase  involves 
definition  of  the  internal  architecture  of  the  software, 
performance  trade-off  analyses  of  algorithms,  specification 
of  individual  programs  or  modules,  specification  of 
interactions  between  components  of  software,  etc.  The 
Design  Specifications  are  important  as  a  means  for  resource 
management  and  quality  control  during  PROGRAMMING. 

*In  the  PROGRAMMING  phase,  software  management  assures  that 
appropriate  technical  choices  are  made,  and  that  high 
quality  workmanship  is  applied  in  producing  code  meeting  the 
Design  Specifications. 

The  PROGRAMMING  phase  often  is  merged  with  DESIGN  and 
so  may  have  no  distinguishable  starting  point.  PROGRAMMING 
involves  final  technical  choices  for  internal  program  design 
and  the  creation  of  programming  language  statements  or 
"code"  that  is  executable  on  the  computer  and  that 
implements  the  design.  Software  management  here  is  the 
aay-to-aay  supervision  of  programmers,  and  guiaing  and 
reviewing  their  technical  results.  PROGRAMwINb  effort 
includes  the  testing  by  each  programmer  of  his  individual 
programs  and  the  preparation  of  pertinent  management  reports 
and  software  aocuments.  Test  reports,  inspection  reports, 
ana  yzo^ram  documentation  act  as  tangible  quality  controls 
over  PKUGRAWhINu  accomplishments. 

*In  the  TESTING  phase,  software  management  assures  that  no 
significant  errors  or  design  discrepancies  will  impede  usage 
of  the  system  m  the  intended  operational  environment. 

The  TESTING  phase  is  concerned  with  verifying  that  the 
software  product  meets  the  functional  specifications  and 
contains  no  significant  discrepancies  or  logical  errors  to 
impede  its  use.  TESTING  proceeds  according  to  a  Test  Plan 
prepared  in  the  DESIGN  phase,  and  focuses  on  the  integration 
of  individual  programmer  results  in  an  overall  working 
system.  Noted  discrepancies  and  errors  are  documented  for 
corrective  action  by  the  programming  team.  Gross 
shortcomings  in  the  original  design  or  functional 
specifications  may  be  exposed;  this  should  lead  to  a  major 
project  review  in  order     to     plan     and     organize  additional 
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definition  and  design  work. 

*ln  OPERATION  of  a  delivered  system,  software  management 
assures  that  maintenance  is  as  well  managed  and  controlled 
as  original  development. 

Tne  OPERATION  phase  follows  the  delivery  and 
installation  of  the  software  in  the  field,  and  involves 
periodic  redesign  and  improvement.  Software  management 
procedures  that  are  recommended  during  development  can  be 
streamlined  to  provide  similar  close  control  of  the  reduced 
effort  committed  to  corrective  design  work.  Thorough 
specifications,  quality  workmanship,  effective  testing,  and 
management  review  remain  highly  important  in  directing 
tecnnical  effort  to  meet  cost  and  schedule  targets. 

♦buring  the  development  period,  the  earlier  activities  may 
oe  continuea  or  re^jeatea  in  oraer  to  remedy  aesign  aefects 
ana  improve  quality  or  tne  final  system. 

Aitnougn  tne  successive  phases  defined  nere  should  be 
loiioweo  whenever  practical,  some  flexibility  is  aavisable, 
especially  wnen  a  project  involves  unusual  complexity  or 
innovation.  For  example,  it  may  be  important  to  carry  only 
a  part  of  the  system  through  to  DESIGN  and  PRO(jRAi'U*iIN(j ,  m 
oraer  to  confirm  feasibility  before  beginning  DEFINITION  for 
tne  entire  system.  Or,  with  limited  effort  available, 
aeveiopment  of  some  supporting  capabilities  or  auxiliary 
programs  may  be  deferred  until  the  mainstream  processing  is 
successfully  completed.  So,  all  parts  of  the  software  need 
not  be  at  the  same  stage  of  development,  and  the  life  cycle 
orientation  may  be  viewed  with  some  flexibility. 


THE  QUALITIES   OF  SUPERIOR  SOFTWARE 


These  general  properties  are  commonly  accepted  as 
characteristic  of  high  quality  software.  The  techniques  of 
software  management  may  be  usea  to  control  quality  at 
acceptaoly  nign  levels  or  to  maximize  quality  insofar  as 
practical  witfi  tne  resources  ana  time  availaole  for  a 
project . 

C0KKt,(^TwL6i3--progr ams  perform  exactly  ana  correctly  all 
tne  functions  expectea  from  the  specifications,  if 
availaole,  or  else  from  the  documentation.  This  means  tnat 
incorrect  aocumen tation  is  as  serious  as  an  incorrect 
program.     Correctness  is  an  ideal  quality,     that     is  rarely 


-13- 


determinable,   so  a  more  practical  quality  is  RELIABILITY. 

RELIAblLITY — Programs  pertorra  without  significant 
detectaDle  errors  all  the  functions  expected  from  the 
specifications  or  the  documentation.  high  reliability 
inaicates  that  programs  are  relatively  trouble  free  m 
performing  what  they  are  claimea  to  do.  t)Ut  an  equally 
important  question  is  whether  tne  functions  ana  performance 
are  adequate  and  suitable  to  a  needed  purpose.  The  latter 
quality   is  callea  VALILITY. 

VALIDITY — Programs  provide  the  performance,  all 
functions,  and  appropriate  interfaces  to  other  software 
components,  that  are  sufficient  for  beneficial  application 
m  the  intended  user  environment.  This  means  the  software, 
without  additional  programming  or  manual  intervention,  has 
the  capabilities  that  reasonably  would  be  expected  for  its 
purpose.  Validity  is  a  quality  of  specifications  as  well  as 
computer  programs.  Examples  of  an  invalid  program  would  be 
an  interactive  editor  that  had  no  online  function  for 
retrieving  stored  text  for  inspection,  or  a  FORTRAN  language 
compiler  that  had  no  DO  loop  implementation.  Validity 
involves  judgement  of  user  requirements,  and  may  change  if 
the  intended  application  or  purpose  is  altered.  Because 
poor  reliability  may  render  a  needed  function  useless, 
reliability  is  necessary  to  validity. 

RESILIENCE  (or  ROBUSTNESS) — Programs  continue  to 
perform  m  a  reasonable  way  aespite  violations  of  the 
assumed  input  and  usage  conventions.  Input  of  unacceptable 
data,  or  an  inconsistent  command,  should  never  cause  a 
result  that  is  astonishing  ana  detrimental  to  the  user--sucn 
as  tne  aeletion  of  any  valia  results  obtamea  previously, 
programs  should  include  routine  cnecKs  and  recovery 
possibilities  that  are  "forgiving"  of  common  user  and  aata 
errors.  Resilience  is  related  to  the  broader  quality  of 
Ui>bat>lLlTY  . 

USeABILITY — Programs  have  functions  and  usage 
techniques  that  are  natural  and  convenient  for  people, 
showing  good  consideration  of  human  factors  ana  limitations. 
For  example,  the  programs  have  few  arbitrary  codes  for  data 
in  input  or  output,  have  consistent  conventions  in  different 
operating  modes,  and  provide  thorough  diagnostic  messages 
for  errors  or  violations  of  use. 

CLARITY — The  functions  and  operation  of  the  programs 
are  easily  understood  from  the  user  manual,  and  the  program 
design  and  structure  are  readily  apparent  from  the  listing 
of  program  statements.  This  means  that  documentation  must 
be  well  written,  but  also     that     the     program     is  carefully 
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aesignea,  witn  meaningful  cnoices  of  variable  names,  use  of 
Known  algoritiims,  frequent  ana  effective  comments  in  the 
program  to  descrioe  its  operation,  and  a  modular  structure 
tnat  isolates  separate  functions  for  examination. 

MAINTAINABILITY — Programs  are  well  documented  by 
manuals  and  internal  comments,  and  so  well  structured  that 
another  programmer  could  easily  repair  defects  or  make  minor 
improvements.  Clarity  is  essential  for  maintainability, 
but  also  implied  are  a  wide  variety  of  good  design 
attributes,  such  as  program  functions  that  help  to  diagnose 
potential  problems,  e.  g.,  periodic  reports  of  status  or 
control  totals;  or,  general  techniques  that  can  be  readily 
adapted  for  change,  e.  g.,  the  isolation  of  constants, 
report  titles,  and  otner   static  data  as  named  variables. 

iHOLiIFlAbILITY--Frogr  am  functions  tnat  might  require 
major  change  are  well  aocumented  and  isolatea  m  distinct 
mouules.  naintainabil 1 ty  is  essential  to  moo  it iaoii i ty . 
out  mod  If laoil ity  means  tnat  a  concertea  erfort  was  maae  to 
anticipate  major  cnanges,  ana  to  plan  the  software  aesign  so 
tnat  tney  coula  oe  maae  easily. 

oLi>*bKALlTi--f rogr ams  perform  their  functions  over  a 
wiae  ranye  of  input  values  ana  usage  moaes.  Programs  are 
not  limitea  to  special  cases  or  ranges  of  values,  when  the 
functions  are  commonly  or  reasonaoly  extenaable  to  a  more 
general  case. 

POKTAbILITY--Progr ams  are  easily  installed  on  another 
computer  or  under  another  operating  system  program, 
btanaard  programming  language  is  used,  and  hardware  or  other 
software  dependent  features  are  isolated  for  easy  change. 

TESTABILITY — Programs  are  simply  structured  and  use 
general  algorithms,  to  facilitate  step-by-step  testing  of 
all  capabilities. 

LFFICILNCY/LCONOMY — Programs  have  high  performance 
algorithms  and  conservatively  use  computer  resources,  such 
as  main  storage,  so  that  the  cost  of  program  operation  is 
low . 


Ui\oAwlZlWU   tuK  ijUFTWAKL  nANAvjLi'ibNT 


*A  project  team  is  tne  basic  organization  for  sottware 
man ag erne n  t . 
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The  team  approach  is  effective  because  a  mix  of  needed 
skills  can  be  quickly  assembled  from  an  organization's 
available  talent,  and  the  most  talented  designers  can  be 
consistently  utilized  on  the  toughest  problems.  Team 
practices  provide  better  oversight  and  counseling  tnrough 
peer  reviews,  better  dissemination  of  tecnnical  and  status 
information  about  software  modules,  and  better  back-up  of 
personnel.  The  project  team  approacn  will  be  an  unusual 
practice  where  computer  personnel  have  been  organized 
according  to  specialties  such  as  "operations",  "system 
analysis"  and  "systems  programming".  Nevertheless,  the 
advantages  gained  by  forming  an  appropriately  skilled  team, 
dedicated  to  one  product  objective,  warrant  a  departure  from 
the  traditional  organization.  A  project  team  may  be  formed 
temporarily  to  carry  out  software  development,  with  further 
effort  after  installation  handled  in  the  normal 
organizational  framework.  However,  the  advantages  of  team 
work  apply  to  maintenance  effort  in  the  OPERATION  phase  as 
well  as  to  the  development  phases.  So  a  project  team  may  be 
advisable  when  major  maintenance  or  improvement  activities 
are  done. 

*For  projects  involving  up  to  about  10  personnel,  one 
individual  should  be  responsible  for  the  entire  system 
design  and  the  project  management. 

This  concept  has  been  called  the  "Chief  Programmer" 
approach,  for  the  individual  must  be  an  exceptional 
programmer  as  well  as  a  very  competent  manager.  Control 
tnrough  one  individual,  especially  from  ulFInITIOn  through 
Tt-STiNb,  offers  a  better  possibility  of  a  consistent,  nigh 
quality  product,  than  would  cooperative  work  oy  several 
individuals  or  smaller  teams.  The  team  chief  personally 
defines  and  designs  the  entire  system,  and  produces  many  of 
the  critical  modules.  The  cnief  exercises  complete 
authority  over  the  technical  results  and  team  production, 
he  or  she  represents  the  project  with  users  or  customers  and 
with  higher  management,  and  decides  the  project  commitments 
made  to  them.  The  chief  controls  personnel  assignments  and 
schedules,  and  has  supervisory  responsibility  for  all  team 
members.  The  team  chief  usually  should  have  a  minimum  of 
five  years  of  diversified,  intensive  experience  in  system 
analysis  and  software  design,  and  should  have  demonstrated 
the  personal  qualities,  such  as  judgement  and  maturity,  that 
are  needed  for  effectively  supervising  other  specialists  and 
for  dealing  with  clients  and  superior  officials. 

*The  project  team  should  include  an  assistant  chief, 
specialists  for  key  problem  areas,  and  supporting  personnel. 
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ine  assistant  team  cniet  substitutes  tor  tne  cniet 
wnenever  necessary,  and  serves  as  his  primary  technical 
aavisor.  ine  assistant  may  personally  oetme,  aesign,  and 
program  some  parts  ot  tne  system.  Ahen  needed,  particular 
specialists  become  team  members  to  handle  difficult  design 
problems  requiring  tneir  unusual  knowledge  and  experience. 
In  PRuGRAwHING  especially,  a  programming  language  specialist 
should  be  available  to  answer  questions  about  the  effect  and 
performance  of  various  instruction  sequences,  and  to  suggest 
better  ways  of  programming  certain  algorithms.  Specialists 
on  program  production  tools,  testing,  and  data  management 
techniques  are  advisable  also.  The  team  secretary  assists 
in  clerical  tasks  aimed  at  improving  the  team  productivity, 
such  as  the  preparation  of  computer  runs,  purchase  of 
computer  supplies,  handling  of  project  records  and  software 
documents.  Additional  support  is  advisable  also  in 
documentation  and  technical  writing,  and  in  administrative 
support  of  tne  team  on  personnel  matters,  training, 
management  information  support,  etc.  Figure  3  depicts  a 
representative  team  structure. 

*ream  techniques  are  appropriate  also  tor  very  large 
projects . 

When  faore  tnan  aoout  ten  people  are  needed  to  uo  a 
project,  one  individual  cannot  effectively  supervise  all  of 
them,  so  tne  design  and  programming  must  be  distriDuted 
among  several  teams.  L>4ever  tneless ,  one  manager  should  oe 
given  authority  over  tne  entire  effort,  anu  snould  maKe  the 
fundamental  decisions  on  worK  oreakdown,  resource 
allocations,  and  software  design.  inis  project  manager  and 
his  subordinate  programming  managers  could  be  viewed  as 
comprising  a  team  to  carry  out  system  design  and  project 
management.  Their  working  procedures  should  be  similar  to 
those  within  a  subordinate  programming  team;  for  instance, 
conducting  periodic  design  conferences  where  each  team  chief 
presents  the  designs  and  other  results  of  his  group.  The 
project  manager  should  exercise  considerable  technical  and 
design  leadership,  and  major  aspects  of  overall  system 
design  would  be  decided  by  his  team.  Support  arrangements 
similar  to  a  programming  team  are  advisable,  as  suggested  in 
Figure  4. 

*Team  techniques  are  applicable  in  managing  contracted 
developments . 

A  team  approach  is  advisable  also  when  the  software 
development  will  be  accomplished  principally  by  private 
contractors.  The  project  manager  or  team  chief  will  serve 
then  as  the  contracting  tecnnical  monitor.  The  contracting 
team,  rather   tnan  designing     and     producing     programs,  will 
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prepare  the  technical  statements  of  work,  and  other 
contractual  requirements  such  as  the  quality  and  performance 
specifications,  and  the  pro3ect  plan  in  regard  to  mandatory 
schedules,  deliverable  products,  and  quality  controls. 
Technical  specialists  in  particular  areas  such  as  testing 
and  performance  evaluation  are  also  necessary  on  the 
contracting  team,  and  special  administrative  support  is  also 
advisable  to  improve  the  quality  and  timely  completion  of 
contract  requirements,  technical  reviews,  acceptance 
testing,  etc.,  as  the  contractors  proceeds  witn  tne 
aeveiopmen t . 

*agency-widfc  direction  ana  evaluation  of  software  policy  and 
quality  control  is  needed  m  addition  to  individual  team 
luanay  emen  t . 

h.n  upper  level  of  software  management  authority  is 
advisable  to  provide  consistent  direction  and  support  for 
individual  teams  with  regard  to  organizational  policies, 
standards,  and  objectives  in  software.  Such  an  individual 
or  office  can  contribute  significantly  in  assessing  software 
technology  for  project  needs,  and  in  reviewing  project 
accomplishments  with  a  view  to  standardizing  team 
procedures,  technical  aids  and  software  production  tools, 
and  even  specific  software  components  that  have  wide  utility 
for  the  organization. 
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irigure  j.     A  representative  Team  urganization 
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Figure  4.     A  Representative  Organization  for  a  Large  Project 
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*;30ftware  problems  originate  most  frequently  in  tne  analysis 
ot  requirements  and  tne  design  of  tne  computer  software. 

Nearly  50%  of  software  flaws  and  errors  exposed  m 
final  testing  or  initial  use  of  a  new  system  are  traceable 
to  shortcomings  and  mistakes  in  the  early  stages  of  a 
project,  i.  e.  from  INITIATION  through  Dt,SIGN.  Most 
authorities  suggest  that  added  effort  in  the  early  stages  is 
the  best  way  to  improve  the  quality  of  software.  The  effort 
up  to  the  start  of  PROGRAMMING  should  comprise  about  40%  of 
the  total  development  effort.  A  greater  allocation  is 
advisable,  provided  working  methods  and  review  techniques 
are  geared  to  strong  improvements  in  the  validity  and  detail 
of  the  Design  Specifications. 

*The  primary  quality  controls  for  the  INITIATION  and 
DEFINITION  phases  are  system  requirements  documents  and  the 
technical  reviews  made  of  them  by  the  parties  concerned. 

quality  control  in  these  early  stages  is  necessarily 
juogementai,  out  will  be  aiaed  considerably  by  stanaardized 
documentation  practice.  Tne  National  oureau  or  Standards 
nas  prepared  guidelines,  FIPS  PUb  38,  tnat  indicate  the 
content  ror  functional  requirements,  project  plans,  and 
otner  software  documents.  Additional  guidelines  ana 
stanaaras  wili  be  issued  m  the  near  future.  The  use  oi 
computer-aided  text  editing  and  document  formatting  will  oe 
especially  helpful  in  rapidly  accommodating  numerous  small 
cnanges  in  specifications  during  repeated  and  lengthy 
rev  lews . 

*otart  a  Project  Record  with  the  INITIATION  pnase  and 
continue  it  over  the  system  lite. 

The  Project  Record  should  be  an  open  record  of  the 
major  decisions  and  of  the  pertinent  documents  and 
specifications.  It  should  be  official,  to  clearly  indicate 
firm  commitments,  yet  openly  available,  so  that  inadequacies 
can  be  detected  as  early  as  possible  and  so  tnat  current 
goals  and  constraints  are  apparent  to  all.  This  record  is 
distinct  from  a  technical  notebook,  which  also  is  often 
effective  as  an  informal  medium  for  exchanging  tentative 
proposals  among  designers  and  programmers. 

*In  the  INITIATION  phase,  identify  all  user  tasks  to  be 
supported  by  the  software  and  the  important  attributes  of 
those  tasKS. 
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The  documentation  of  the  INITIATION  phase  should 
characterize  the  working  environment  in  which  the  proposed 
software  will  operate.  This  would  include  a  description  of 
user  tasks  to  be  supported,  the  present  methods  and  computer 
aids  for  carrying  out  these  tasks,  and  the  present 
limitations  that  the  proposed  system  would  overcome.  The 
description  should  be  oriented  for  the  users'  review,  so 
that  they  may  identify  omissions  or  shortcomings  of  the 
proposed  software  solution. 

♦Maintain  a  continuing  interaction  with  the  users  and  their 
managers  during  the  development  period. 

The  potential  users  or  their  managers  are  authorities 
on  the  working  environment  m  which  tne  proposed  system  must 
tunction.  They  can  provide  vital  information  on  the 
acceptaole  performance  and  design  goals  for  the  system,  such 
as  input  data  characteristics,  toleraole  response  time,  etc. 
Their  strong  support  and  involvement  increases  tne 
likelihood  of  successful  planning  and  design,  and  helps  in 
oDtaming  needed  requirements  data  and  evaluations  of  the 
t>roposed  system.  A  highly  recommended  approach  is  to  have 
selected  users  serve  as  members  of  the  project  team,  not 
necessarily  to  perform  design  worK,  but  to  help  to  decide 
and  to  document  requirements.  Project  milestones  should 
include  scheduled  review  sessions  as  the  requirements  and 
alternative  solutions  are  evolved,  to  obtain  user  comment 
and  concurrence. 

*S tr aightf oward  automation  of  existing  manual  systems  may 
not  yield  the  greatest  benefits  from  computers. 

The  computer's  capability  to  do  complex  processing  of 
large  volumes  of  data  at  high  speed  generally  means  that 
computers  can  provide  better  analytical  assistance  to  human 
decision-makers  than  could  any  number  of  clerical  people. 
The  way  in  which  data  records  are  organized  for  manual 
handling  also  may  not  be  the  best  to  use  in  computer  storage 
and  retrieval.  The  INITIATION,  DEFINITION  and  DESIGN  phases 
should  include  research  toward  new  algorithms  and  data 
handling  procedures  that  allow  the  computer  to  serve  as  more 
than  a  high-speed  clerk  and  file  cabinet.  For  example, 
there  are  mathematical  procedures  that  can  greatly  improve 
inventory  controls,  scneduling  for  scarce  resources,  and 
plans  tor  performance  of  complex  tasks.  Also,  there  are 
well  proven  computer  techniques  that  provide  rapid  retrieval 
of  the  very  data  needed  for  a  human  task,  rather  than 
requiring  people  to  manually  hunt  through  long  computer 
print-outs  of  all  related  data.  Improved  algorithms, 
carefully  chosen  for  each  application,  can  increase  human 
and  computer  productivity  by  many  times. 
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*Kadicai  aavances  in  automation  carry  exceptional 
r isK--exper imen t  with  new  approaches  through  prototype 
software,  ana  oe  preparea  to  throw  away  one  or  more  systems 
oetore  confirming   the  design. 

Too  often,  the  cause  for  software  management  failure  is 
overambition  or  executive  zeal  to  bring  the  benefits  of 
automation  to  a  pedestrian  manual  operation  in  one  great 
leap  forwara.  Never  undertake  to  produce  in  a  single 
project  step  a  system  for  which  there  is  no  technological 
precedent.  Keep  the  scope  of  systems  within  the  bounds  of 
your  experience  with  successful  pr o j ec ts--ad vance  your 
system  capability  in  modest  increments. 

*use  consultants  ana  visit  other  organizations  to  identify 
potential  t'roblems  in  building  a  system  and  to  verify  that 
system  oojectives  are  attainable. 

^jystems  tnat  nave  any  possibility  or  challenging  tne 
bta te-ot-the-ar t  must  be  planned  cautiously.  uesearcners, 
consultants,  ana  organizations  wno  nave  pioneered  an 
innovative  type  or  system  can  help  aetine  tne  major  pitfalls 
ana  now  tney  may  oe  avoioea. 

stematically  evaluate  available  software  of  a  comi->arable 
t^pe,  wnetner  commercial  or  other-user  developed,  oerore 
maKing  a  ririii  commitment  to  develop  a  new  system. 

ine  feature  analysis  technique  is  very  useful  for 
evaluating  availaole  software,  and  comparing  proposed  system 
requirements  against  the  current  state-of-  the-art.  Feature 
analysis  proceeds  by  decomposing  the  requirements  into 
inaividual  functions  ana  attributes,  and  then  developing 
snort  aescriptions  of  what  the  available  software  provides 
tor  each  feature.  The  descriptions  should  be  organized  in  a 
table  for  easy  comparison.  Available  software,  if  adequate 
to  the  need,  can  usually  be  purchased  for  much  less  than  the 
cost  of  producing  similar  programs.  Even  if  new  development 
is  required,  existing  software  may  serve  for  some  components 
and  reduce  overall  system  cost. 

*i:ormulate  ana  evaluate  alternative  solutions  tnat  differ  m 
tne  risK  anu  cost  of  development. 

ir'ruaent  management  requires  a  looK  at  other  choices 
Detore  aecidmg  upon  one  system  to  be  oought  or  built, 
bmce  software  cost  ana  risK  are  now  so  important,  tney 
naturally  are  Key  cons laer at ions  m  conceiving  of  competing 
at^proacnes.  ine  possiDility  of  meeting  tne  need  tnrougn 
commercial  camputmg  services  or  other  sources  of  availaole 
sortware  certainly  should  oe  presentea  as     one  alternative. 


It  feasible.  No  definite  guidelines  can  be  offerea  on  how 
to  formulate  alternatives.  Clearly,  thorough  knowledge  of 
the  state-of-the-art  is  essential  and  will  indicate 
candidate  solutions  that  are  more  or  less  innovative. 

♦Estimate  the  future  system  costs  by  several  techniques  and 
adopt  a  conservative  projection. 

There  is  no  well  formulated  cost  estimation  method  that 
is  highly  favored  and  widely  used.  Various  published  data 
exist  on  programmer  production  rates,  but  different 
variables  are  involved  and  some  important  factors  are 
omitted  in  one  case  or  another.  BROOKS  (see  References) 
gives  a  concise  summary  of  published  data.  YOURDON  and 
WALSTGN  also  have  data  showing  the  gains  possible  with 
recent  innovations  such  as  structured  programming. 
Experience  and  judgement,  however,  are  the  most  accepted 
means  to  reach  an  overall  recommendation  on  the  time  and 
funding  needed.  Published  data  should  certainly  be  used  as 
a  guide,  noting  carefully  the  pertinent  factors  included. 
Examine  costs  of  coraparaole  systems  and  advertised  costs  of 
commercial  packages  for  added  eviaence.  Do  a  detailed  cost 
oreakdown  of  the  system  by  major  components  and  by  major 
technical  activities  or  work  processes.  Include  operation 
and  Development  costs,  to  have  a  full  life  cycle  investment 
comparison  of  alternative  systems.  Above  all,  recognize  the 
uncertainty  of  any  early  estimate  of  cost,  and  choose  a 
conservatively  high,  yet  reasonable  figure.  Reevaluate  the 
cost  estimate  periodically  as  the  development  proceeds  and 
better  definition  of  the  software  evolves. 

*aave  the  intended  users  review  and  approve  the  requirements 
and  recommended  solution  defined  in  the  INITIATION  phase. 

User  concurrence,  along  with  any  evaluations  and 
recommendations  from  the  user  review,  should  be  in  the 
record  concluding  the  INITIATION  phase.  This  could  be  as 
simple  as  initials  in  a  project  notebook  for  a  very  small 
project,  or  may  require  an  official  report  and  approval 
letter  for  a  large  project.  To  reach  their  assessment, 
users  should  be  given  a  cost  and  benefit  analysis  of 
alternative  systems,  in  addition  to  the  general 
specification  of  each. 

♦Resist  generalizing  the  system  capability  to  accommoaate 
requirements  that  cannot  oe  resolved  before  DESIGN  begins. 

If  an  imposea  schedule  will  require  starting  DESIGN 
before  ail  requirements  are  defined,  a  judgement  must  be 
made  of  wnether  the  uncertainty  allows  useful  effort  to 
proceed.       If  it  aoes,   then  the  unknown  requirements  must  be 
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anticipated  and  the  most  likely  possibilities  defined.  The 
impact  of  these  alternatives  on  the  system  design  must  be 
determined,  and  if  great  enough,  it  may  be  necessary  to 
proceed  with  definition  and  design  of  several  systems  until 
the  unknowns  are  eliminated.  Be  cautious  about  generalizing 
the  required  system  capability  to  handle  arbitrary,  unknown 
oeraands.  (generalized  systems  often  involve  great  risk  in 
aevelopment,  and  may  have  poorer  performance  and  efficiency 
tnan  more  constrained  and  specialized  designs,  be  forceful 
in  investigating  potential  needs,  m  order  to  resolve 
requirements  questions  and  uncertainties  as  early  as 
possible . 

*Use  engineering  analysis,  modeling  or  simulation,  and 
experience  data  to  establish  the  performance  needed  in 
critical  software  operations. 

Software  designers  often  assume  that  necessary 
performance  can  be  attained  by  "tuning"  programs  once  they 
are  running.  On  the  other  hand,  early  decisions,  for 
instance  on  the  hardware,  the  programming  language,  or  the 
principal  algorithms  to  be  used,  could  be  so  poor  as  to 
eliminate  any  hope  of  reaching  a  needed  level  of 
performance.  Engineering        analysis,        coupled  with 

measurements  from  comparable  existing  software,  should  be 
used  to  investigate  the  expected  system  performance  as  a 
function  of  the  attainable  performance  for  basic  processing 
steps.  Performance  goals  should  be  specified  for  the 
critical  operations,  to  support  feasibility  judgements  and 
to  serve  as  validation  criteria  for  the  DESIGN  and 
PKOGRAMMINCj  phases.  Early  decisions  that  constrain  the 
design  or  implementation  approach  for  the  software  should 
snow  clear  evidence  that  the  necessary  performance  would  be 
achievaole . 

♦Include  witn  tne  Functional  Requirements  the  performance 
and  quality  objectives  that  the  delivered  software  must 
meet. 

ihe  specification  of  quality  criteria  should  begin  in 
tne  ubFlNlTION  phase  in  order  to  have  a  basis  for  judging 
tne  subsequent  design.  Use  the  definitions  given  above  as  a 
starting  point,  to  be  expanded  and  specialized  for  a  given 
project.  For  example,  identify  the  system  functions  that  are 
most  subject  to  future  modifications  and  the  nature  of  the 
required  raodif labil ity .  As  another  example,  describe  such 
useability  characteristics  as  the  processing  points  where 
recovery  procedures  are  required  and  the  acceptable  form  of 
recovery.  Include  the  quantitative  performance  goals  for 
principal  algorithms  and  processing  steps.  In  the  Project 
Plan,     define     the     technical     reviews    and     the  evaluation 
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procedures  tnat  wiii  oe  used  to  assure  tnat  tne  quality  and 
performance  oDjectives  are  met  througnout  tne  project. 

♦Develop  tfie  Functional  Requirements  in  a  structured  format, 
to  expedite  review  and  validation. 

A  free-form  narrative  approach  to  documenting 
requirements  will  make  it  difficult,  except  for  very  simple 
programs,  to  isolate  individual  functions  and  performance 
figures  for  later  confirmation  of  design.  A  better  approach 
is  first  to  decompose  overall  requirements  into  major 
functional  groups,  such  as  data  entry  or  report  generation, 
or  into  processing  modes,  such  as  online  query  or 
transaction  auditing.  Then  describe  individual  requirements 
under  each  group  with  a  separate  statement  of  a  function 
that  must  be  provided  or  a  performance  or  design  attribute 
that  must  be  achieved.  Each  requirement  should  include 
criteria  for  confirming  its  achievement,  the  testing  or 
evaluation  methods  involved  m  confirmation,  and  any 
comments  or  discussion  that  amplify  the  requirement 
statement  or  snow  its  origin. 

*Do  not  conclude  the  DhFIwITIUN  pnase  without  having  an 
approved  specitication  tnat  incorporates  cnanges  and 
reiinements  emerging  from  user  reviews. 

wever  proceed  into  detailed  design  when  ambiguity  or 
Known  errors  exist  in  the  requirements  specifications, 
basic  disagreements  are  more  liKely  to  lead  to  tundamentdl 
redesign  than  to  minor  and  easily  rendered  change. 

♦Formulate  concrete,  measurable  milestones  of  progress  for 
the  design,  programming,  testing,  installation,  and  Initial 
operation  of  the  system. 

It  is  impossible  to  exercise  control  and  take 
corrective  action  without  concrete  evidence  of  where 
problems  lie.  This  means  that  progressive  milestones  must 
be  conceived  covering  the  life  of  a  project.  They  must  be 
events  and  accomplishments  that  are  directly  observable  by 
the  software  manager,  such  as  the  receipt  of  a  specified 
document,  the  successful  operation  of  a  given  program,  or 
the  conclusion  of  a  scheduled  review  meeting.  They  must  be 
sufficiently  frequent  over  time  to  serve  as  progress 
indicators  or  trouble  alerts  if  not  completed.  The 
dependence  of  later  milestones  on  earlier  events  shoula  be 
revealed  by  drawing  up  a  graph  or  network  relating  all 
milestones.  Progress  during  DESIoiNi  and  PKOGRAWWING  should 
De  measured  by  requirea  events,  such  as  reviews  or 
Specifications,  and  not  Dy  programmer  estimates  ot  percent 
of     code     completea.       The     accompanying     cnart    describes  a 
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minimum  recommended  set  of  milestones  over  a  system's  life 
cycle . 

♦Establish  controls  and  milestones  tnat  will  identify  and 
correct  errors  or  omissions  as  early  as  possible. 

The  cost  of  correcting  an  error  or  adding  capability  to 
a  system  increases  significantly  as  a  project  proceeds.  by 
the  time  system  testing  begins,  errors  are  over  ten  times 
more  costly  to  fix  tnan  if  tney  naa  been  recognized  in 
aesign.  Adaitional  time  given  to  improving  specifications 
ana  to  tnorougnly  critizing  tnem  is  wortnwnile  in  reducing 
later  t^roject  costs. 

*Pian  at  least  of  total  project     effort     for  management 

requirements  ana  tecnnical  reviews. 

when  tne  neeaea  effort  nas  oeen  underest imatea ,  wnicn 
too  often  IS  the  case,  careful  reviews  ana  management 
analysis  may  be  thrown  asiae  to  put  all  effort  into  design 
and  programming.  Obviously  this  will  worK  to  the  detriment 
of  quality  objectives,  ana  probably  will  result  in  even  more 
time  required  for  testing  and  rework  later.  Make  sure  that 
project  estimates  include  the  time  and  effort  for  detailed 
review  of  work  as  it  progresses,  and  for  continuing  review 
of  project  schedules  and  resource  expenditures.  Insist  on 
using  this  time  as  intended,  so  that  project  status  and 
problem  sources  can  always  be  accurately  defined. 

♦Plan  contracts  for  design  and  production  only  after  tne 
detailed  Functional  Requirements  and  Project  Plan  have  been 
prepared  ana  approved. 

Contractors  cannot  be  selected  or  effectively  monitored 
without  the  quality  assurance  and  validation  criteria  that 
are  emooaiea  in  these  aocuments.  They  also  proviae  the 
first  practical  oasis  for  aecomposing  the  development  effort 
into  worK  packages  that  different  contractors  mignt 
unaertaKe.  contract  requirements  particularly  snouia  call 
tor  stanaaraizea  aocumen tation  ana  coorainatea  tormat  among 
tne  tunctional  Kequirements  and  the  succeeding  aesign 
specifications,  to  aia  in  tracing  and  correlating  results  as 
tne  eifort  proceeds. 
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figure  b.     Chart  ot  Kecommended  i'tinimum  hilestones 
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Description  of  Milestones 


1.  Initiate  Project  Record  with  plan  for  feasibility  study. 

2.  Complete  analysis  of  user  tasks  to  be  supported. 

3.  Complete  definition  of  alternative  software  solutions. 

4.  Complete  feasibility  study  report  with  recommended  solution. 

5.  Obtain  user  concurrence  with  recommended  solution;  budget  approval 

for  project.  ,  ■ 

6.  Complete  Functional  Requirements. 

7.  Complete  Project  Plan. 

8.  Complete  user  validation  of  Functional  Requirements. 

9.  Complete  software  architecture  and  definition  of  major  modules. 

10.  Complete  Test  Plan;  and  Design/Coding  Standards. 

11.  Complete  Design  Specification  of  all  component  programs;  tools  definition, 

12.  Complete  validation  of  Design  Specification. 

13.  Complete  unit  testing;  accept  all  modules  for  source  code  control. 

14.  Initiate  fault  reporting  procedures. 

15.  Complete  integration  testing. 

16.  Complete  delivery  and  shakedown. 


GUIDELINES   FOR  DESIGN  AND  PROGRAMMING 


*The  second  major  source  of  problems  lies  in  the  quality  of 
programs  as  they  are  produced. 

About  40%  of  software  errors  are  traceable  to 
misinterpretations  of  specifications,  errors  in  designing  to 
them,  or  mistakes  m  writing  program  statements.  Recent 
research  to  improve  software  management  has  concentrated  on 
these  problems.  A  variety  of  new  techniques,  such  as 
structured  programming  ana  automated  quaiit;/  controi,  are 
Deginning  to  come  into  widespread  practice,  but  much  more 
must  be  aone.  misinterpretation  of  specifications  would  be 
reduced  by  more  formal  or  matnematical  specification  methods 
rather  tnan  exclusive  use  of  narrative  English.  Analytical 
metnods  of  design,  replacing  tne  subjective  criteria  used 
neretoiore,  would  reduce  tne  frequency  of  programming  errors 
and  costly  maintenance.  Software  managers  must  Keep  current 
with  such  developing  areas  in  software  engineering  research. 
Continuing  training  programs,  special  in-house  worksnops, 
and  experimental  projects  are  good  avenues  for  stimulating 
the  computer  staff  toward  improving  their  working  practices 
and  technical  knowledge. 

*0n  beginning  the  DESIGN  phase,  prepare  an  explicit 
specification  of  the  quality  design  and  programming 
standards  that  must  be  observed  in  the  DESIGN  and 
PROGRAMMING  phases. 

The  Functional  Requirements  should  be  accompanied  by 
definitions  of  mandatory  quality  objectives,  that  serve  as  a 
starting  point.  Using  these,  a  list  of  design  attrioutes 
can  be  specified,  extending  in  detail  to  conventions  on  the 
use  of  programming  language  features  and  constraints  for 
individual  program  design.  This  expansion  of  quality 
specifications  therefore  will  encompass  the  Coding  Standards 
to  be  used  in  the  project,  and  will  provide  a  framework  for 
starting  design  of  individual  modules.  For  example,  the 
quality  standards  may  define  the  extent  of  diagnostic 
messages  to  oe  provided,  the  types  of  data  errors  to  oe 
detected  uy  validation  algoritnms,  the  manner  of  isolating 
particular  processes  subject  to  future  modification,  etc. 

♦Develop  the  Design  Specifications  as  a  hierarchy  of 
processes,  specifying  the  input  and  output  of  each  process, 
and  the  data  flow  interconnections. 
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A  fiierarchy  is  an  ettective  means  to  decoiTipose 
tunctionai  requirements  into  successively  more  aetaiiea 
operations.  ine  approacii  is  well  matched  to  wnat  is  called 
"top-down"  desiyn  tor  software,  wnicn  proceeds  Dy  successive 
retinements  ot  general  functions,  until  arriving  at  the 
component  modules  or  programs  that  must  De  implemented  to 
Duiia  tne  system.  Documentation  tor  this  tunctionai  design 
approacn  is  rather  simple,  and  it  portrays  the  overall 
system  operation  in  an  easily  followed  graphical  way.  The 
References  include  books  ana  reports  on  the  technique. 

♦Complete  the  Design  Specifications  before  beginning  to 
coae . 

It  is  vital  that  the  design  be  completely  worked  out 
before  programming  begins,  so  that  programming  effort  is  not 
wasted  on  erroneous  design  notions  and  so  that  programming, 
when  started,  has  a  complete  guide  to  assure  quality  work. 

♦Decompose  the  system  into  program  modules  that  are  simple 
and  easily  understood,  usually  meaning  about  100  program 
statements  in  size. 

The  advantages  of  small  modules  are  many.  Their 
functions  are  more  clearly  and  simply  describable.  This 
helps  direct  the  programmer  toward  a  clear  objective.  Their 
brevity  improves  clarity  and  speeds  review  for  validity  and 
quality.  ir'rogress  may  be  more  accurately  evaluated  when  the 
units  of  programming  work  are  small  ana  readily  understooa. 

♦Define  general  service  moaules  to  reauce  duplication  m 
program  modules. 

In  designing  the  system  into  modules,  keep  track  of 
replicated  functions  ana  aefine  modules  that  provide  such 
functions  as  services  that  all  programs  may  use.  These 
service  modules  usually  are  those  that  manipulate  standard 
data  structures  in  a  system  of  programs,  for  example, 
entering  items  m  tables,  retrieving  items,  managing  storage 
allocation,  etc. 

*Use  standard  high  level  programming  languages. 

The  use  of  vendor -un ique  programming  language, 
especially  machine  or  assembly  language,  should  be 
prevented,  except  for  very  small  portions  of  code  where 
assembly  language  may  be  essential  to  achieving  the 
performance  goals  of  the  software.  These  exceptional  cases 
should  be  strictly  controlled  by  explicit  approval.  Keep  a 
standard  high  level  language  version  of  assembly  programs 
for  documentation  and  maintenance  assistance. 
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*Use  structured  programming  tecnniques. 

Structurea  programming  is  a  concept  that  encompasses 
programming  team  management,  design  methods,  and  programming 
technology.  Numerous  books  and  articles  are  available, 
describing  the  design  and  programming  techniques  especially. 
Published  results  from  evaluation  projects  confirm  that 
increases  in  programmer  productivity  and  reductions  in 
program  errors  are  achieved,  often  as  factors  of  two  or 
greater.  Investment  in  the  training  and  management 
orientation  needed  to  practice  these  methods  will  be 
wor  thwhile . 

*laentity  or  develop  programming  aids  to  increase  team 
proauctivity ,  ana  issue  guidelines  to  the  team  tor  using 
them . 

ipeciiic  r ecommenaations  on  programming  tools  are  given 
later.  selection  of  tools  ana  specifications  tor  unique 
tools  neeaea  lor  tne  project  snoula  oe  aone  as  early  as 
possible  in  tne  ut-hlNITlQw  ana  DtiiGN  phases. 

*^^onauct  weeKly  reviews  by  tne  programming  team  of 
individuals'  worK. 

inis  practice  helps  all  team  members  to  Keep  informed 
of  the  design  and  production  as  it  proceeds,  and  may  act 
also  as  a  training  opportunity.  But  primarily  it  focuses 
the  combined  talent  of  the  team  on  the  quality  of  the  team 
output  as  a  whole.  Errors  or  inferior  design  can  be  more 
quickly  exposed  than  with  reviews  by  a  single  manager.  The 
results  of  each  review  session  should  be  documented  as  a 
guide  for  the  individual  programmer  to  improve  or  correct 
his  work.  In  structured  programming,  these  review  sessions 
are  called  "structured  wal k-throughs" ,  to  suggest  that  each 
programmer  leads  the  team  through  the  design  and  operation 
of  his  module   in  the  same  pattern  as  it  was  developed. 

*ijesign  anJ  program  tor  quality  before  performance. 

oefore  programmers  make  any  effort  to  tune  their 
moaules  for  improved  performance,  all  quality 
cnar acter istics  required  shoula  be  realized.  wecessary 
comments  ana  program  arrangements  tor  clarity, 
maintainability,  moo  it laoil i ty ,  generality,  ana  all  requirea 
lunctions  snould  be  completea  ana  contirmea   in  reviews. 

*tor  large  projects,  conauct  inspections  at  successive 
stages  of  aesign  ana  production,  and  make  all  necessary 
changes  before  work  moves  to  tne  next  stage. 
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In  addition  to  team  reviews  of  each  programmer's 
progress,  more  formal  inspections  are  recommended  tor  large 
projects,  to  examine  subsystems  or  closely  related  modules 
produced  Dy  different  programming  teams.  Tnese  inspections 
siiould  serve  as  progress  milestones  in  tne  Project  Flan, 
rne  inspection  team  may  include  experts  outside  of  tne 
teams,  in  addition  to  tne  team  cniefs  and  tne  responsiole 
programmers.  Tne  inspection  team  leader  (or  moderator) 
snoulu  not  nave  oeen  a  participant  in  producing  tne 
subsystem  under  review  ano  must  nave  the  authority  to  inform 
nigher  management  of  negative  findings.  A  team  size  of 
about  ten  persons  maximum  is  aovisaole,  and  separate  reviews 
snould  be  made  when  design  is  complete,  when  code  and  unit 
tests  are  complete,  and  when  first  integration  tests  are 
comple  te . 

*Use  Iterative  review  techniques  for  validation. 

Validation,  also  sometimes  called  verification,  means 
confirmation  that  a  system  is  reliable,  and  meets  its 
specifications  as  well  as  the  requirements  of  the  user 
environment.  Team  reviews  or  inspections  should  always 
trace  design  specifications  back  to  functional  requirements, 
taking  note  of  findings  from  past  reviews.  In  this  way,  the 
logical  thread  that  stems  from  the  original  specification  is 
followed  to  the  working  software,  ano  successively  examined 
for  continuity  and  possible  shortcomings.  Reviews  should 
evaluate  the  suitability  of  the  software  for  the  intended 
use,  and  whether  the  necessary  quality  is  being  achieved. 

*in  the  FKUG RAM WING  phase,  the  individual  programmer 
performs  "unit  testing". 

unit  testing  determines  that  the  individual  programs 
work  m  isolation  from  otners  and  meet  the  specifications 
given  with  the  assignment.  unit  testing  must  oe  performed 
according  to  standard  criteria,  ratner  tnan  ad  noc  methods 
contrived  by  each  programmer.  The  programmer  provides  test 
reports  for  use  in  review  and  inspection  tasks.  Testing 
criteria  are  described  in  the  guidelines  tor  TbSTINb,  which 
follow . 

*Put  program  modules  under  source  code  control  when  unit 
testing   is  completed. 

Completion  of  unit  testing  signifies  internal  project 
delivery  of  a  module  for  integration  with  others  in  the 
overall  software  system.  Source  code  control  restricts 
further  program  changes  to  formally  approved  changes  that 
have  been  decided  from  an  overall  system  viewpoint  as 
necessary     to     meet     project     goals.     Source  code  control  is 
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neeaea  tor  best  use  of  programming  effort  in  the  terminal 
period  of  tne  development  schedule. 

*Before  the  DESIGN  phase  is  complete,  a  formal  test  plan  for 
tne  system  should  be  approved. 

The  Test  Plan  should  be  started  in  the  DEFINITION 
phase,  for  the  Project  Plan  must  account  for  the  effort 
required  for  the  testing  strategy  to  be  used.  Some  details 
on  the  number  of  tests,  test  inputs,  etc.  cannot  be 
specified  until  the  design  is  nearly  complete.  The  Test 
Plan  describes  the  TESTING  phase  effort  and  the  testing 
schedule,  and  defines  the  standard  test  set,  i.  e.  the 
complete  range  of  test  cases  that  will  be  repeatedly  used  to 
assure  reliaoility  of  the  delivered  software.  The 
development  of  this  standard  test  set  is  a  significant 
effort  that  must  be  completed  by  the  start  of  TESTING.  The 
test  cases  must  be  provided  by  manual  calculations  or  other 
independent  means,  so  that  the  correct  output  expected  is 
Known  beforenana.  other  means  may  include  program 
analyzers,  simulations,  taoles  oi  ranaom  numbers  or  function 
values,  results  of  comparable  systems,  etc.  The  aeptn  of 
tne  lest  i'lan  is  inoicative  of  the  reliability  tne  user  may 
expect  11  testing  is  successfully  completed.  laeally,  the 
Test  i'ian  is  aeveloped  by  a  tecnnical  group  tnat  is 
inoepenaent  of  tne  designers  and  programmers,  ano  that  can 
require  tests  that  focus  on  perceived  weaK  points  m  the 
design.  users  should  be  represented  in  test  planning,  and 
tne  final  plan  should  oe  jointly  agreed  oetween  users  and 
the  chief  designer. 


GUIDELINES   FOR  TESTING 


♦Delivered  software  for  large  systems  typically  has  errors 
at  a  rate  of  one  m  every  3\diu  program  statements. 

wnen  iEoiii>4«o  oegins,  tne  errors  present  may  be  three  or 
more  times  nigner  than  tnis  rigure.  Testing  is  the  process 
of  executing  programs  with  representative  input  data  or 
conditions,  tor  wnich  tne  correct  results  are  known,  to 
determine  wnether  incorrect  results  occur.  Testing  provides 
information     that  helps  to  isolate  errors.     Debugging   is  tne 
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subsequent  work  of  a  programmer  to  explicitly  identify 
errors  and  to  correct  them  by  changes  to  program 
instructions  or  data.  Many  large  program  systems  never 
become  error-free,  i.  e.  "correct".  Instead,  they  are 
maintained  and  managed  m  terms  of  the  number  of  known 
errors  that  are  allowed  to  exist--i.e.  successive  versions 
or  releases  are  planned  to  reduce  the  known  errors  to  some 
acceptaole,  but  perhaps  still  nign  rigure. 

*iesting  is  the  only  practical  way  to  detect  program  errors 
so  that  they  may  oe  eliminateo. 

Testing  only  determines  if  errors  are  present.  It 
cannot  verify  tnat  no  errors  exist  in  a  program.  bucn 
confirmation  is  conceivaole  m  only  two  ways.  One  is  to 
exnaustively  test  tne  program  on  every  possible  input.  Lven 
for  simple  programs,  this  could  take  millions  of  years  even 
If  done  automatically  by  a  computer.  The  second  conceivable 
way  IS  to  prove  correctness  as  a  logical  and  mathematical 
theorem  about  the  program  instructions.  This  approach  is  a 
current  research  topic  in  computer  science,  and  proofs  have 
been  done  manually  for  simple  programs  up  to  a  few  hundred 
instructions.  Research  work  is  now  aimed  at  simplifying 
proof  requirements  through  programming  practices  suited  to 
this  objective,  and  through  computer  theorem-proving 
software  that  would  reduce  the  complex  mathematical  effort 
involved.  No  break-through  is  predicted  in  the  near  future 
that  would  make  proof -of -cor r ec tness  practical  ana 
economical  for  the  average  programmer  to  use  on  normally 
complex  programs. 

*Adopt  specific  criteria  to  govern  unit  testing  of  all 
programs. 

wany  aifterent  testing  criteria  are  conceivable,  out 
most  nave  no  direct  relationship  to  possible  errors  ana 
aesign  flaws,  ana  so  proviOe  no  information  to  improve 
reliaoility.  For  example,  one  could  require  a  speciric 
number  of  test  runs  per  program.  but  unless  eacn  test  was 
appropriately  distinct,  no  more  benefit  would  be  obtained 
tnan  from  running  only  one  test.  A  ranaom  sample  among  most 
or  a  program's  input  values  may  be  an  effective  criterion, 
but  It  IS  useful  only  if  the  valid  input  values  are  small  m 
number,  so  tnat  a  significantly  large  number  of  random  tests 
can  be  generated  and  run  within  the  time  and  resources 
available.     Other  possible  criteria  are: 

a)  test  every  instruction  at  least  once; 

b)  test  every  branch  of  the  program; 
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c)     test  every  ioyicai  path  within  tne  program 
witn  at  most  one  iteration  or  every  loop. 

(A  logical  patn  is  a  sequence  ot  instructions,  some  possioly 
repeatea,  trom  input  to  termination  ol  a  program.)  ine  above 
tnree  criteria  are  successively  more  stringent;  they  are 
preferable  to  many  others  because  they  are  well  airectea  at 
possible  faults,  and  tend  to  be  efficient  m  using  test 
ef f or  t . 

Recommended  testing  guidelines  that  are  broadly 
applicable  to  software  cannot  be  simply  stated.  The  subject 
is  technically  complex  and  beyond  the  scope  of  this  guide. 
The  National  Bureau  of  Standards  is  currently  preparing  a 
guideline  that  will  include  results  from  current  research  to 
develop  testing  theory  and  to  lay  down  broadly  effective 
principles.  The  most  important  consideration  at  this  time 
is  to  avoid  arbitrarily  chosen  tests  by  each  programmer, 
that  do  not  clearly  exercise  previously  untested  functions 
ana  logic. 

*bcneaule  progressive  tests  to  build  up  to  a  representative 
full   system  test. 

Testing  snoula  begin  with  discrete  tests  of  oasic 
system  runctions  tnat  are  neeoea  in  oraer  to  go  more  complex 
ana  realistic  tests.  Simple  input,  output,  ana  control 
functions  are  examples.  Keeping  each  test  restricted  to  a 
given  function  allows  the  results  to  clearly  identify 
problems  encounterea.  ultimately,  tne  software  should  be 
exercised  oy  composite  test  cases  that  represent  full  system 
operation  and  expected  usage  conditions.  Composite  tests 
Should  be  conceived  to  stress  the  software  performance.  The 
limits  on  such  factors  as  accuracy,  acceptable  volume  of 
data,  or  number  of  concurrent  users,  should  be  verified  if 
included  in  the  specifications.  The  diagnostic  features  of 
tne  software  should  be  exercised  as  well  with  test  cases 
having  invalid  input.  The  Test  Plan  should  encompass  most, 
if  not  all,  of  the  capabilities  defined  in  the  Functional 
Requirements . 

*Use  program  analyzers  to  assure  that  all  program  functions 
have  been  exercised. 

A  program  analyzer  is  a  couiputer  prograai  that  collects 
data  about  another  program,  including  data  about  it  as  it 
operates.  in  particular,  an  analyzer  can  determine  whether 
test  inputs  nave  caused  all  the  program  instructions  to  oe 
used.  This  verifies  that  all  capabilities  have  been 
exercised,  and  identifies  extraneous  cooe  that  may  have  no 
purpose  and  should  oe  removed. 
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♦Experience  indicates  that  testing  usually  requires  about 
40%  of  available  development  time. 

Careful,  disciplined  design  should  reduce  the  need  for 
extensive  testing,  but  more  theoretical  development  and 
experience  with  improved  techniques  will  De  needed  to  assure 
this  IS  always  so.  Certainly,  better  specifications  would 
eliminate  rework  that  sometimes  is  initiated  during  TESTIMG 
when  it  becomes  clear  that  the  design  was  incomplete  or 
haphazaraly  contrived.  but,  it  seems  best  to  be 
conservative  in  project  scnedules  as  well  as  cost  estimates, 
so  allow  a  major  portion  of  project  time  ror  testing  and 
uesign  rework. 

*Use  a  fault  reporting  process  to  manage  deougging  and 
testing . 

It  IS  good  practice  to  have  a  formal  report  system  by 
which  Detected  errors  ana  discrepancies  are  recorded  and 
fully  described.  These  reports  will  help  to  confirm  that 
all  known  errors  are  fixed  before  delivery.  They  also  help 
to  trace  multiple  instances  of  the  same  anomalous  behavior, 
so  that  debugging  assignments  can  be  made  for  related 
problems  and  the  debugging  effort  reduced. 

♦Conduct  regression  tests  after  program  rework  has  been 
done . 

The  TESTING  phase  is  primarily  concerned  with 
integration  testing,  i.e.  tests  to  determine  whether 
different  program  modules,  produced  rather  independently, 
will  work  together  correctly.  /(hen  errors  are  found  and 
eliminated  by  debugging,  new  undetected  errors  may  be 
introduced.  Regression  testing  means  that  previously 
successful  tests  are  rerun  to  detect  any  introduced  errors. 
A  complete,  successful  run  of  the  entire  standard  test  set 
should  be  accomplished  on  tne  software  oefore  concluding 
iE^Dl'lNG  and  delivery  to  the  customer. 

♦Supplement  integration  testing  with  system  validation 
reviews. 

Although  specifications  are  expected  to  be  of  nigh 
caliber  when  PKUbRAmniNG  begins,  practical  use  of  the 
resulting  system  during  TESTIwG  may  reveal  that  original 
notions  of  desired  operation  were  fallacious.  Reviews  of 
test  sessions  should  consider  the  general  suitability  of  the 
system  as  well  as  strict  adherence  to  discrete  functional 
specifications.  Any  discrepancies  should  be  considered  for 
rework,  rather  than  delivering  a  system  that  in  the 
judgement  of  users  and  designers  will  not     be     adequate  for 
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ttie  user's  t^urpose. 

*lesting  snouid  assure  a  niyn  measure  ot  reiiaoiiity  tnat 
wouia  oe  expectea   in  operation  ot  computer  programs. 

Correctness  is  tne  aesirea  quality  tor  software,  but 
also  IS  an  ideal  condition  that  otten  is  not  economically 
attainaole  or  contirmaole  for  complex  software.  So,  tne 
term  " r el laDil i ty "  is  useful  to  denote  the  lacK  of  detectea 
errors  with  respect  to  some  basis  of  observation. 
Observations  could  be  made  over  a  long  time  perioo  of 
repeated  use  of  the  software,  and  reliability  then  defined 
as  the  ratio  F/N,  with  F  the  number  of  detected  faults,  and 
N  the  number  of  runs  or  trials  of  the  program. 

Software  reliability  measurements  will  reflect  how  the 
software  is  being  used,  as  well  as  the  faults  or  errors  it 
contains.  If  routine  usage  is  limited  to  previously 
debugged  functions,  reliability  will  apparently  be  high.  If 
usage  changes,  reliability  may  suddenly  degrade  seriously, 
as  previously  unexercised  code  is  used  and  its  errors 
revealed . 

Reliability  measurement  is  a  recent  subject  of  software 
engineering  research.  i*iore  results  are  needed  before  a 
minimum  acceptaole  figure  can  be  recommenaea.  The  most 
effective  use  of  reliability  oata  m  project  management  also 
IS  not  certain  until  more  project  experience  is  at  nana. 

*k'ian  a  snaKeaown  period  atter  delivered  software  is 
installea . 

A  shakeaown  period  may  be  dictated  by  hardware  alone  if 
a  new  computer  system  is  involved,  but  this  kind  of  trial 
use  may  also  effectively  supplement  the  previously  performed 
integration  testing  of  software.  The  shakedown  scenario 
should  be  as  close  to  the  actual  anticipated  usage  as  costs 
and  procedures  permit,  but  do  not  depend  upon  the  shakedown 
operation  for  any  operational  service.  Use  the  fault 
reporting  process  and  records  of  shakedown  run  time  to 
measure  and  evaluate  reliability.  The  shakedown  period 
needed  to  attain  stable  and  reliable  operation  may  be 
several  months,  particularly  if  hardware  installation  is 
extended  over  this  time  and  considerable  user  training  is 
being  done. 
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AVOIDING  ENDLESS   SOFTWARE  PROBLEMS 


*Use  the  management  techniques  for  development  during  the 
OPERATION  phase  as  well. 

The  programming  and  design  ettort  available  during 
OPERATION  IS  generally  reduced  compared  with  tne  development 
phases.  however,  the  goals  of  software  inanagement  ao  not 
change;  indeed,  maintenance  costs  may  dominate  the  total 
life  cycle  cost  of  a  software  product.  Maintenance  worK  on 
software  is  still  a  design  cnallenge,  but  one  constrained  by 
tne  program  structure  ana  instructions  alreaoy  in  use. 
unless  technically  controllea,  maintenance  may  aegraae 
reliaoility  ana  otner  qualities  tnat  were  initially  present, 
maintenance  tenas  to  aisorganize  and  complicate  tne  prograiii 
structure,  increasing  the  potential  for  errors  ana  tne  cost 
of  future  cnanges.  It  should  not  oe  assuaiea  that  the 
existing  software  is  a  sufficient  constraint  to  prevent 
unsupervised  maintenance  from  Decerning  costly  or  haphazara. 
Maintenance  should  be  conducted  under  a  tight  management 
regime  that  includes  oefinition  ana  scheauling  of  manageable 
work  assignments,  preparation  and  review  of  specifications, 
team  review  of  design  and  code,  standard  tests,  and 
documen  tat  ion . 

*Keep  a  failure  and  discrepancy  reporting  process  to 
continually  manage  software  reliability. 

Failure  reports  are  normal  in  the  TESTING  phase  and  the 
initial  period  of  OPERATION,  but  they  continue  to  be 
important  as  maintenance  continues  over  the  years.  These 
reports  are  needed  for  sottware  managers  to  accurately 
aetermine  if  maintenance  work  is  being  accomplished  as 
assigned  and  if  greater  testing  ana  rework  effort  is 
warranted.  be  cautious  about  proposed  improvements  tnat  are 
not  strongly  supportea  by  users  or  inbicated  by  problem 
r  epor  ts . 

*Evaluate  software  periodically  ana  plan  improvements  to 
avoia  oDsolescence  ana  to  minimize  future  conversion  costs. 

ine  nigh  investment  in  current  software  ana  tne  high 
cost  of  replacement  often  leaa  to  i^rolongea  use  or  sortware 
designs  that  nave  become  obsolete.  Obsolescence  may  oe  aue 
to  cnanging  technology  or  economic  factors,  or  to  changes  in 
organizations  ana  worK  loads.  Major  software  subsystems 
should  be  evaluatea  periodically,  to  identify  mod ir icat ions 
that  would  eliminate  operational  limitations,  or     woula  use 
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improvea  naraware  or  sottware  tecnnolo^y.  wajor  improvement 
m  an  existiny  system  may  oe  economically  aone  tnrou^jn 
progressive  stayes  ot  reworK.  equally  important,  rework  to 
stanaaroize  or  restructure  component  routines  may  increase 
tne  leasioility  ot  translerin9  tne  sottware  directly  to  a 
luture  computer  com  i>j  ur  a  t  ion .  bucn  prospects  empnasize  tne 
importance  ot  standard  documentation,  oecause  tne 
replacement  ot  modules  or  interlaces  would  oe  impractical 
witnout  clear  oerinition  ot  tne  existing  sottware  design. 

*use  configuration  management  metliods  to  control  sottware 
status . 

Configuration  management  involves  close  control  ot 
operational  software  modifications,  to  assure  tnat 
continuing  service  is  not  adversely  affected  by  cnanges  and 
tnat  the  changes  made  are  sufficiently  important  to  justify 
their  cost  within  the  total  maintenance  effort  available, 
proposed  changes  should  be  agreed  to  jointly  by  tne 
programming  group  and  the  users,  and  scneduled  with  the 
appropriate  priority  as  part  of  the  total  worKload  of  the 
programming  team.  Changes  should  first  be  made  to  a  test 
copy  of  the  software,  not  tne  same  copy  used  tor  everyday 
service.  Thus  testing  can  proceed  without  delaying 
operational  service,  and  only  a  fully  cnecKed  out  system  is 
delivered  tor  use. 

♦i'er  lod  ically  review  the  t^roject  kecord  and  current 
documentation,  to  insure  tnat  maintenance  results  are  oemg 
recorded  tor  later   use  oy  other  programmers. 

rtaintenance  responsibility  is  sometimes  misconstrued  as 
a  personal  prerogative  that  may  be  done  with  eacn 
programmer's  style,  unless  management  exerts  its  autnority 
tor  quality  control  to  meet  tne  organization's  neeos. 
Liocumen  tat  ion  is  a  tedious  but  crucial  chore,  and  it  is 
inexcusable  for  operational  sottware  to  oe  altered  witn  no 
evidence  except  in  the  code  itself.  As  in  all  phases  ot 
software  worK,  management  action  should  be  as  immediate  as 
possible.  The  team  review  process  is  recommended  for  it  may 
remove  any  appearance  of  management  inquisition.  weeKly 
meetings  that  include  reviews  of  documentation  updates  may 
De  sufficient  to  keep  project  information  current  and 
compl e  te . 


-39- 


THE   NEED  FOR  SOFTWARE   PRODUCTION  TOOLS 


♦Software  tools  are  computer  programs  that  automate  tasks  in 
the  management,  production,  and  testing  of  software. 

A  wide  variety  of  tools  have  been  developed  and  used  in 
the  software  field,  but  few  standard  tools  are  recognized  as 
applicable  to  every  project.  The  most  widely  used  tools  are 
the  compilers  for  high  level  programming  languages,  that 
transform  program  statements  into  executable  machine  codes, 
inany  other  tools  can  appreciably  reduce  ttie  tedium  of 
programming  tasks  ana  can  expedite  tne  application  of 
standaras  to  a  programmer's  products.  Surveys  of  computer 
installations,  as  well  as  recent  project  experiences,  show 
tnat  tne  value  of  errective  tools  is  not  wioely  appreciated, 
ana  an  aaecjuate  repertoire  of  effective  tools  is  selaom 
t>rovided.  Tne  importance  of  good  software  proauction  tools 
is  confirmea  over wnelmmgly  by  all  recent  experience  ana 
autnor ita t ive  research  in  software  engineering. 

*t^roviae  interactive  computer  support  for  programming  on 
every  project. 

Developing  programs  and  documentation  on  an  interactive 
computer  facility  significantly  improves  the  efficiency  and 
working  conditions  for  the  programming  team.  Programmers 
can  concentrate  on  technical  problem  solving  without  being 
limited  and  burdened  by  scheduling  of  computer  runs,  as  in  a 
batch  computer  support  facility.  Tentative  design  ideas  can 
be  experimentally  programmed  and  evaluated  in  a  short  time, 
allowing  the  development  of  a  quality  design  to  move  along 
rapidly.  Various  debugging  and  testing  tools  can  be  used  to 
snorten  the  time  for  identifying  sources  of  errors.  A 
separate  interactive  support  facility  is  advisable  when  the 
operational  computer  is  unsuitable  for  interactive  usage 
because  it  is  intended  for  a  batch  or  real-time  application. 
The  technology  and  economics  of  interactive  support  software 
have  been  amply  proven,  and  the  availability  of  oasic 
systems  is  adequate  for   tnis  general  recommendation. 

*use  preprocessors  to  supplement  a  standard  nign  level 
programming  language. 

A  preprocessor  accepts  programming  statements  written 
in  an  improved  aialect  ol  a  stanaara  progr amiiiing  language, 
ana  translates  tnem  into  stanaara  language  acceptaole  to  a 
compiler  program.  Tne  merit  of  a  preprocessor  is  tnat 
restrictive  conventions  that  still  exist  in  stanaard 
languages,       sucn      as      card     column     orientation,     can  be 
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circumvented  tor  improving  programmer  productivity  and 
reaucmg  errors,  without  losing  tne  transportability 
advantages  ot  tne  standard  language.  For  example, 
structurea  programming  coula  De  applied  even  witn  t'UKXKAi^, 
wnicn  is  innerently  unsuitea  tor  it.  t<qually  important, 
many  or  tne  coaing  stanoaras  ot  a  pro3ect  coula  de  cnecKed 
tor  eacn  program  oy  a  preprocessor,  as  a  means  ot  automated 
quality  control.  However,  a  preprocessor  may  oe  aitiicult 
to  use  together  witn  a  symoolic  aeougger ,  oecause  the 
ueougger  output  would  rerer  to  tne  preprocessor  output 
statements,  ana  so  aeougging  results  woula  oe  aitticult  to 
correlate  witn  the  source  program  statements. 

*t,stablish  a  standara  liorary  ot  specialized  sottware  tools. 

Software  tools  generally  are  inexpensive,  otten 
available  trom  other  users  for  the  cost  of  reproduction,  ana 
sometimes  concise  and  easily  modified  for  unique  project 
needs.  Tools  are  usually  specific  to  one  programming 
language,  and  they  perform  a  limited  set  of  functions  on 
programs  written  in  that  language.  For  each  language  m  use 
then,  a  basic  set  of  tools  can  be  assembled,  with  each  one 
assisting  in  an  important,  distinct  design  and  programming 
task . 

*Use  an  editor,  program  librarian,  and  program  analyzer  as 
standard  tools. 

An  editor  assists  the  programmer  in  making  numerous 
Changes  ana  aaditions  to  a  program,  without  the  burden  of 
tilling  out  coding  forms  or  manipulating  format  aetails,  ana 
with  less  risK  ot  errors  in  repetitive,  trivial  changes.  A 
gooa  editor  will  automatically  perform  routine  tormattmg 
ana  stanaara ization  actions,  ana  witn  prompting  ana  cnecKing 
facilities  incluaea,  it  can  prevent  or  laentity  trivial 
syntactic  errors  that  a  programmer  may  maKe. 

a  program  librarian  is  a  necessary  complement  to  an 
eaitor,  ana  proviaes  for  storing  on  the  computer  ana 
retrieving  all  program  modules  and  distinct  versions  of  them 
that  may  aevelop  auring  a  project.  The  usual  tile  system  of 
an  interactive  computer  facility  may  serve  as  an  adequate 
program  librarian.  Features  that  are  useful  ana  often 
provided  in  special  tools  for  this  purpose  include 
capabilities  to  time  and  date  stamp  different  modules,  to 
employ  special  naming  conventions  for  programs,  to 
efficiently  store  related  program  versions  by  storing  only 
the  relative  changes,  etc. 

A  program  analyzer  is  software  that  scans  the  source 
statements     in     another  program  to  collect  information  about 
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tnem  or  to  provide  tor  data  collection  about  tne  program 
behavior  wfien  it  is  executed.  A  variety  ot  analyzers  may  be 
useful,  tor  special  formatting  of  programs,  for  auditing 
adherence  to  coding  standards,  for  extracting  cross 
reference  and  data  usage  reports,  for  performance 
measurement,   for  test  monitoring,  and  similar  purposes. 

Many  tools  of  these  types  are  available,  and  special 
tools  are  often  economical  to  develop  for  a  project. 

*(Jse  a  simple  project  information  system  to  maintain  data  on 
schedules,  milestones,  and  resources  used  on  distinct  tasks 
and  activities. 

Simplicity  is  the  key  to  keeping  the  management  and 
administrative  effort  within  bounds  tor  small  projects.  For 
large  projects,  fiscal  and  contractual  requirements  may 
dictate  extensive  record  Keeping,  but  it  is  still  advisable 
tor  technical  management  to  nave  simplified  project  data 
that  may  expose  basic  problems,  ratner  than  to  oe  buried 
under  all  possiole  details.  It  a  computer  mtormation 
system  is  used,  it  should  maintain  a  gross  networK  ot  major 
milestones  and  produce  a  tew  briet  reports  on  manpower  and 
other  expenditures  tor  tnese  milestones  and  otner  support 
act IV 1 ties . 


STANDARDS   FOR  QUALITY  CONTROL 


A  number  of  distinct  quality  control  techniques  are 
known  to  be  effective,  and  some  have  been  briefly  described 
in  the  preceding.  In  all  cases,  there  is  no  documented 
standard  of  practice  for  the  technique,  but  there  are 
published  articles  and  sources  ot  experience  to  help  m 
developing  appropriate  standards  for  an  interested 
organization.  A  summary  definition  of  known  techniques  is 
given  oelow. 


PROJECT  RLCORL/--An  official  journal  ot  project  plans, 
specifications,  scnedules,  worK  assignments,  oudgets, 
technical  standards,  etc.  that  serves  to  inform  all 
concerned     parties     ot  the  goals,  worK  methoos,  nistory,  and 
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status  of  the  effort. 

ir'ROJECT  PLAi\--a  detailed  plan  of  milestones,  resource 
allocations,  ana  quality  control  procedures  to  oe  used  in 
tne  aevelopment  phases  tor  a  software  product.  incorporated 
in  tne  ir'roject  Kecora  ana  updatea  as  neeaea  during 
aevelopiuen  t . 

L/ULUi-iLi.'^'iaiiow  i)TAwijAKDo--unif  orm  requirements  on 
uocuiiients  for  tne  project  ana  the  software,  tnat  aetine 
scope  ana  identification  of  each  aocument,  as  well  as  lormat 
anu  content,  ana  that  cover  tne  principal  specifications  ana 
tecnnical  aescr ipt ions ,  ootn  internal  to  the  project  ana 
aeliveraole  witn  tne  sortware. 

AUTuwATLb  DUCUi*ibwTATIOi.M  AlDb--Computer  software  for 
text  editing  ana  document  formatting,  and  for  automatea 
preparation  of  program  documentation  such  as  ilow  charts  or 
cross  reference  listings  of  aata  elements  ana  program 
s ta temen  ts . 

STRUCTURED  SPECIFICATIONS — The  approach  of  coordinating 
all  specification  documents  in  format  and  content,  and  using 
formal  or  mathematical  specification  techniques  to  reduce 
aiTibiguity.  This        facilitates       successive  validation 

milestones  as  increasingly  detailed  specifications  are 
developed.  The  series  of  specifications  involved  begins 
with  the  feasibility  study  description  of  system  concept, 
continues  with  the  functional  requirements,  and  extends  to 
the  individual  program  module  specifications. 

wuALIT:^  and  i^ERFORMANCb  RE(^UlKEMENTS--Augmen ta t ion  of 
tne  functional  requirements,  to  state  the  specitic  quality 
attriDutes  expected  of  the  software  and  the  quantitative 
performance  necessary  m  Key  operations  or   test  cases. 

:3PELlFICATI0w  CuwTkOL--A  formal  process  of  reviewing 
and  at^proving  proposed  changes  to  specifications,  with 
consideration  of  tneir  tecnnical  validity,  impact  on  project 
resources  ano  scneauie,  and  impact  on  worK  completed  or  m 
progress.  Tne  goal  is  to  inhibit  continuing,  capricious 
Changes,  and  instead  stabilize  the  specifications  on 
distinct  deliverable  versions  of  the  software  that  can  be 
managed  for  production. 

STRUCTURED  pRUGRAMMIwG — Technical  practices  and 
criteria  governing  particularly  the  programming  techniques 
and  language  usage  for  increasing  programming  productivity 
and  reducing  program  errors. 
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Tup-L»uwiM  PKUOKAw  utVhLUPiyibw'i--'i'ne  approach  ot 
implemen tincj  and  testing  program  systems  oy  Duiiding  program 
modules  starting  witn  those  at  the  most  general, 
application-oriented  level  (the  "top"),  and  proceeding 
successively  down  to  the  most  specialized,  computer- 
dependent  level    (the  "bottom"). 

DiiSIblM  and  CODING  STAI\IDARDS--De tailed  specifications  of 
generally  applicaole  design  requirements  based  on  quality 
oDjectives,  including  such  programming  conventions  as 
indenting  and  spacing  of  program  statements,  use  of 
comments,  naming  of  variables,  use  of  language  features, 
etc . 

SOURCE  CODE  CONTROL — The  process  of  labeling  and 
archiving  program  modules  accepted  for  formal  technical 
control,  thereby  limiting  future  programmer  modifications  to 
formally  reviewed  and  approved  changes. 

TEAM  kEVIEAS--Techn ical  team  conferences  to  audit 
design  proposals,  specifications,  and  completed  programs  for 
qual 1 ty . 

UNIT  FuLD£-Rb--The  programmer's  technical  records  of 
design  and  testing  worK  on  individual  program  modules, 
sometimes  callea  "unit  development  folders".  As  standard 
uocuments  in  a  project,  these  augment  the  project  record  and 
specifications  oy  providing  more  technical  oocumen tation  as 
necessary  for  reviews  and  inspections.  Especially  important 
m  large  projects  subject  to  frequent  personnel  changes  or 
r eassignmen ts . 

INbPECTIONS--Formal  review  and  auditing  of  subsystems 
or  groups  of  program  modules  at  design,  code,  and  test 
stages,  by  an  independent  technical  team.  Applicable  to 
larger  projects. 

STANDARD  PRODUCTION  TOOLS — Computer  software  to  aid  the 
programming  team  in  creation,  documentation,  testing,  and 
analysis  of  the  deliverable  programs.  These  would  include, 
for  example,  the  standard  programming  language  compiler, 
program  librarian,  test  data  generator,  symbolic  debugger, 
etc . 

PROGRAM  ANALyZERS--Sof tware  tools  for  automated 
analysis  of  program  modules,  to  confirm  complete  adherence 
to  coding  standaras  ana  quality  criteria,  ana  to  assure 
thorough  testing  according   to  standards. 

PkuJElt  INtORMATION--Kecora mg  ana  analysis  ot  the 
allocation     and     expenaiture     of     team     effort     on  different 
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modules  ana  milestones  for  the  pro-gram  system,  and  on 
various  production  activities,  in  order  to  monitor  progress 
and  to  support  best  practical  resource  management. 

TESTINb  S1ANDARDS--Spec if ic  criteria  governing  tne 
program  testing  to  De  performed,  to  assure  that  programs  are 
unirormly  tested  oy  all  programmers  and  tnat  all  program 
instructions  and  capaoilities  are  tested  to  the  extent 
pr  ac  t ical . 

iNuLi^LwubiNT  itibT  irLAiNS--use  Of  independent  tecnnical 
st^ecialists  to  prepare  compr enensive  tests  of  tne  integrated 
soitware  sj-stem. 

t\rtuLi  Kbir'UKi  1 rormal  process  for  documenting 
oDservea  errors  or  discrepancies  m  program  operation  ano 
capaoility,  to  aid  oeougging  and  reliaoility  assessment,  and 
to  validate  sottware  design. 

CunFIvjUkAi  lUN  hAhAGLwENT--A  formal  process  of 
controlling  the  operational  software  status  and  reviewing 
proposed  improvements,  to  Dest  direct  maintenance  effort  and 
to  minimize  impact  of  maintenance  and  testing  on  operational 
ser V  ice . 


FITTING  CONTROLS   TO  THE   PROJECT  SIZE 


Software  projects  differ  tremendously  m  complexity, 
acceptable  scnedule,  and  available  resources.  More  than 
anytning,  tne  available  personnel  determine  wnat  technical 
goals  are  acnievaole  and  wnat  quality  controls  may 
reasonaoly  De  imposed.  Figure  6  illustrates  a  Dasic 
judgement  of  where  the  above  controls  are  best  utilized  m 
regard  to  project  size.  Tne  controls  tnat  are  perceived  to 
oe  more  demanding  of  tecnnical  or  supporting  effort  are  not 
recommended  for   smaller  projects. 
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Figure  6.     Chart  of  Recommended  quality  Controls 
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FINAL  HOMILIES 


A  few  simple  cautions  may  help  to  avoid  unqualified 
failure  in  undertaking  software  development,  given  present 
Knowleoge  of  now  best  to  do  it. 

Cnoose  reasonable  goal s .  Although  computer  technology 
has  amazing  capaoil i ties ,  we  remain  primarily  limited  by 
human  creativity  m  regara  to  software  acnievemen ts . 

nave  a  de  tailea  plan  ♦  Im^jortant  tasKs  may  be  forgotten 
or  expectations  may  expana  unless  all  is  recoraeu  as  a 
reminuer . 

tut  one  per  son  m  control ,  Tnere  snoula  be  strong 
leaaersniiJ  towara  one  well  unoerstooa  concept. 

use  gooa  tools .  i^roauctive  tools  are  tne  only  certain 
way  to  accelerate  nuiuan  accompl  isnmen ts . 

Cater  to  the  sof twar e  user .  The  user  judgement  is  tne 
ultimate  answer  on  success  or   failure  of  tne  product. 

build  up  in  steps .  Confidence  from  successful 
achievement  of  modest  goals  will  lead  more  quickly  to 
grander  oojectives. 

admit  mistakes  early .  Pursuit  of  the  hopeless  leads 
only  to  further  despair--poor  choices  and  misconceptions 
require  drastic  remedies  as  early  as  possible. 

Consider  star  ting  over .  Patcning  a  poor  result  has 
limiteo  return;  a  wiser  action  may  be  to  start  fresn  at  the 
oeg  mn  ing  . 
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